HiLexed 2 released
HiLexed is a top-down, unlimited lookahead parser system. It uses EBNF as grammar notation and separates syntax from semantics. It is a system to dynamically build parsers. On first invokation, sematic actions are compiled into bytecode, but if sematics change, these actions are dynamically recompiled again. There is no separate compilation step.

[UPDATE] I am making progress with regards to recursion. I now think that HiLexed will be able to support all forms of recursion, including left and hidden recursion. Unfortunatly this means that I will have to hold off publishing the midway version mentioned here earlier. With regards to tree-building, the only difficulty I see is handling types properly (no casting allowed), due to dynamic compilation. Anyway, one problem at a time.

Have a look at the files test.syntax and test.semantics included in the download for an example of how to parse some of the more difficult parts of a java-like language. The example includes an unbounded lookahead problem (java modifiers) as well as other forms of local ambiguity problems (methods, properties, static initializers and instance initializers). For clarity, the language described by the grammar only contains constructs that I consider difficult to parse using other parsers.

HiLexed's lookahead engine needs no left-factorization and no help from user-defined syntactic predicates. Unlimited lookahead can be determined statically for non-ambiguous grammars, without a combinatorial explosion.


I have a back-burner project of collecting difficult parsing problems, so please feel free to contact me if you have one. I mostly work on the problem of computer language parsing, but if your problem is from some other area, I am still interested in hearing about it.

So far my list only contains three items: the classic C++ "property or not" ambiguity, deciding between multiple recursive productions and of course Ruby's recursive and context sensitive? "here docs".


Turtles : BASIC for the JVM
Includes HiLexed v1.x, a compiler and libraries needed to produce and execute java binaries from the BASIC dialect Turtles. You can also run your programs in interpreted mode.

BASIC recursion : "Turtles all the way down". *


As a compilation test and development target, QuiteBASIC's implementation of the classic game Pong was used. Big thank you to Nikko Ström for kindly allowing me to use this game's source code from the QuiteBASIC website to demonstrate the new compiler.

Please feel free to try the game (now running on a java platform near you!)