Token language

The tokens include <identifier>, <number>, <reserved word>, <symbol>, <string>, and <meta statement>.  They are defined as regular expressions as follows.

<letter> --> a | b | ... | y | z | A | B | ... | Z | underscore
<identifier> --> <letter> (<letter> | <digit>)*

<digit> --> 0 | 1 | ... | 9
<number> --> <digit>+

<reserved word> --> int  | void | if | while | return | read | write | print | continue | break | binary | decimal

<symbol> --> left_parenthesis | right_parenthesis | left_brace | right_brace | left_bracket | right_bracket | comma | semicolon | plus_sign | minus_sign | star_sign | forward_slash | == | != | > | >= | < | <= | equal_sign | double_and_sign | double_or_sign
(The symbols are: ( ) { } [ ] , ; + - * / == != > >= < <= = && ||)

<string> --> any string between (and including) the closest pair of quotation marks.
<meta statement> --> any string begins with '#' or '//' and ends with the end of line ('\n').