Rust backend for Lark

[erezsh]

I've been experimenting with implementing LALR in Rust, using the same algorithm and architecture that Lark uses.

Early tests show a x20 speed up. i.e. the same text+grammar parse in Python in 60 seconds, and 3 seconds in rust!

The benefit of using the same architecture is that it should be easy to make it into a plugin for Lark, that allows users to keep the same Python interface as now, but get a huge speed boost.

I imagine it will end up more in the 5x-10x speed-up range, due to the Python interop, but that's still quite a bump.

How does it sound?

Is anyone here interested in such a project?

[nchammas]

I wonder if the maintainers of LLGuidance would be interested.

[erezsh]

I would say unlikely, they aren't using the actual Lark implementation, they're only using the Lark syntax for grammars, and with some modifications too.