Here are some places to start with, some of his "famous" mailing list posts: https://news.ycombinator.com/item?id=17329902 and 12 year old reddit bullet list of his too: https://old.reddit.com/r/programming/comments/badl2/luajit_2...

Baseless speculation - was it because the VM was so small and well documented?

Never did program much in Lua but really enjoyed the parts where they describe the virtual machine and instruction set.

EDIT:

Link to the paper I mean https://www.lua.org/doc/jucs05.pdf

For a layman like myself this is very approachable. Again I cannot think of another VM language that has anything this small and well documented.

There was a great thread on lambda-the-ultimate where Mike and a some other famous JIT people talk about the benefits of different approaches. I highly recommend searching for those threads.

I’ll trawl my archives and maybe have a Christmas present for you.

Lua naturally avoids hashing strings as much as possible. Short strings always have their hash precomputed and local variables are internalized. The default method for implementing variable scopes in dynamic languages just uses a lot of hash tables, but Lua cleverly avoids this:

https://github.com/lua/lua/blob/master/lobject.h#L549

https://github.com/lua/lua/blob/master/lstring.c#L185

Not a language designer, but I have been told the big thing holding back Python is that it exposed too much of its internal guts to external APIs. With so much of the internals exposed, it becomes difficult to make some required backend changes.

Not the whitepaper, but here's some resources that I saved for myself to read that might be helpful.

https://www.freelists.org/post/luajit/How-does-LuaJITs-trace...

http://lua-users.org/lists/lua-l/2011-02/msg00742.html

I think I remember the whitepaper, but don't know where to find it these days.

Elisp bytecode now has a compiler using libgccjit, in Emacs 28.

LuaJIT simply did a lot of heavy lifting, implementing a full blown compiler with register allocation and all of that. Lua wasn't special that way. Rather, the Python devs didn't care that much about speed, and Python has the baked in notion that Python programs are a tree of nested dictionaries. Thus, there is only so much performance that could be wrung out of PyPy. But it still helps a lot.

Python 3 was of course a tragedy. If they were going to break stuff at all, they could have made some of that dynamicity optional (few programs use it). Then PyPy could have been the reference implementation and it would be crazy fast, eliminate the GIL, and so on. Just like Lisp implementations have done for 50+ years.