A Lex and Yacc Front-End for Python

Lex

• A (jflex compatible) Lex specification for Python will be constructed by adapting Grammar/Tokens from the Python source distribution.
• some details may be derived from this
• Indent/dedent tokens will require special treatment, as will tabs.
• UTF-8 in Lex is discussed in this including the following lex macros. We won't worry about this in version 1 but likely will look into it in a subsequent version.

  ASC     [\x00-\x7f]
  ASCN    [\x00-\t\v-\x7f]
  U       [\x80-\xbf]
  U2      [\xc2-\xdf]
  U3      [\xe0-\xef]
  U4      [\xf0-\xf4]
  
  UANY    {ASC}|{U2}{U}|{U3}{U}{U}|{U4}{U}{U}{U}
  UANYN   {ASCN}|{U2}{U}|{U3}{U}{U}|{U4}{U}{U}{U} 
  UONLY   {U2}{U}|{U3}{U}{U}|{U4}{U}{U}{U}
  

Yacc

• A Yacc grammar for Python will be constructed by adapting Grammar/Grammar from the Python source distribution
• See also this documentation