LuaJIT limits both the number of registered callbacks [1], and re-entering the interpreter from a JIT'd callback [2] (it will try to detect and prevent JIT for such functions, but if that fails you get a panic).
[1] http://stackoverflow.com/questions/31042530/why-does-luajit-...
[2] http://stackoverflow.com/questions/25924755/c-and-lua-unprot...