Lezer

marijnhaverbeke.nl

188 points·llimllib··5 comments

Very interesting read. I’ve written a number of syntax highlighters and have experienced nearly all these problems. I like the way codemirror currently does it (I basically am able to define my own parser in code and just pass the info CM needs), but will give Lezer a try. Ive found declarative forms like used by sublime and textmate to get you 80% of what you need in 1/10 the time, and then it takes 20/10 time to get that last 20%. One thing I do hope is that the new codemirror be in typescript, which would make it self documenting. Using the current code base feels a bit outdated, though it is so small and well done relatively that it’s still a joy to use.

I believe it will be in Typescript and, if you look at his last couple of posts, he mentions building a documentation generator for Typescript code specifically for this project.

”In a backtracking system, you never know when you've definitely parsed a piece of content—later input might require you to backtrack again
[…]

A GLR parser can split its parse stack and run both sides alongside each other for a while until it becomes clear which one works out.”

So, depth-first vs breadth-first, where the former is a bad idea, performance-wise, because the ‘better’ parses may be found late in the game, so that, early on, very few branches can be cut soon?

[regret]

And (as you know, if you've read the linked article) nondeterministic parsing is not fast.

Reply on news.ycombinator.com