Lua VM running in a WASM environment
github.com
github.com
Copy this: https://gist.github.com/anonymous/af755d7cf0f71a574f39f547d3...
And paste it into the VM: https://cdn.rawgit.com/vvanders/wasm_lua/d68f46a8/main.html
It's a modified version of Lumen (https://github.com/sctb/lumen) made to run on this VM. It defines a runtime, a reader, a compiler, then compiles the runtime from Lisp to Lua and prints out its own source code.
Lumen is also capable of compiling to JS, and I wanted to show it printing out its own source code as JavaScript instead of Lua. But unfortunately I was getting some strange WASM-related memory errors when compiling as JS, so I decided to settle for a neat demo that everybody can run.
(The demo could be a little less cryptic, and I could do a better job of explaining it, but it's late.)
I love how multi-paradigm Lua is, we used to do component based composition with metatables but I've seen OO inheritance and a slew of other approaches as well.
- https://github.com/giann/fengari - new project, most exciting IMO
- https://github.com/daurnimator/lua.vm.js - emscripten based, has best interactions with DOM (but is a dead end)
- http://www.moonshinejs.org/ - no longer maintained, interesting model though
- https://github.com/paulcuth/starlight - transpiler: you loose ability to use proper coroutines which are IMO most interesting feature for lua in browser
- https://github.com/creationix/brozula/ - unmaintained
- fengari: Dunno, nice logo & name though :) Not done yet?
- lua.vm.js: Pretty good, but as you say in your README: "Next step is to iterate on the Lua <=> JS interoperability. Clever solutions to the lack of finalisers in Javascript are being searched for." (-> leak mem if passing stuff to JS in certain situations)
- Moonshinejs: paulcuth/gamesys' previous effort. It's a Lua VM written in JS, that accepts just Lua 5.1 bytecode. Works in IE6+. Very small. Nice 'DOMAPI' plugin; nice in the sense it mostly just works, and when it doesn't, the plugin is so small & simple you can hack it yourself.
One thing to note is when getting strings back from the DOM using DOMAPI, I've run into situations where the strings when accessed Lua-side are in fact 0-indexed not 1-indexed. (the_string .. "") is a working normalization technique.
Coroutines work very well here.
Also, do not overlook the in-browser debugger addon. It's a seriously nice piece of work, that more than makes up for the lack of source map support.
It deals with the lack of finaliers in Javascript by providing explicit .retain() and .release() mechanisms.
- Paulcuth/Starlight: As you say, transpiler, no coroutine support though.
How to get coroutines: ... I was looking into this, and you need something like using kripken's Relooper algorithm or a more general version of Facebook's Regenerator. Scary stuff.
Note about both moonshinejs & starlight, with respect to the string library: It implements Lua patterns by turning them into JS RegExp. The implementation isn't good enough as it stands; it's easy to get the wrong results, as the escaping logic is wrong. Damned if I know where though. I tried to fix it and just broke a different class of patterns...
- Brozula: Dunno
- Lua5.1.js: Dunno. Would be nice to have a better API for interaction than the Lua C API. No commits since 2013, like Brozula.
- Lua.js (lua2js): Dunno. Several open issues about e.g. '<' operator not working. No commits since 2013. And:
_lua_coroutine["resume"] = _lua_coroutine["running"] = _lua_coroutine["status"] = _lua_coroutine["wrap"] = _lua_coroutine["yield"] = _lua_coroutine["create"] = function () { not_supported(); };Lua is a unique language and it's ease of embedding made it a perfect fit for WASM.
Long term, I would love to see a Lua to WASM compiler/transpiler with additional Lua runtime functionality efficiently implemented in WASM as well. Since WASM already has its own stack-based VM, why not skipthe additional Lua VM bytecode execution layer?
I would love to work on this myself, but right now the time investment isn't practical for me.
Maybe I should move on to something a little more practical
Note that there seems a lot of misinterpretation of WASM as a good VM bytecode target. You're stuck with i32/i64/f32/f64. Strings have to be manually managed through byte indexing. There's no GC. Targeting WASM directly is really close to targeting an actual ISA directly. So efficient implementation to WASM is going to come off closer to something like LuaJIT. It also doesn't facillitate self modifying code so lazily compiling will imply either using a trampoline to jump between multiple WASM modules or recompiling everything
There's no jump instruction in WASM, so arbitrary goto has to be converted into loop-and-switch. The Befunge VM I linked breaks that down to only relooping between basic blocks. Browsers optimize that pattern. There's room to play around & see if they can optimize something like a CPS VM. Instead of dispatching based on the value of a memory cell, encode an instruction as a 'store-next-instruction to local, break to instruction implementation (if not short bytecode), have that instruction end by looping back, where we switch to next instruction which repeats the store-next-instruction to local'. Where next-instruction encodes this dance repeatedly. Hot areas of the VM may then be inlined by the browser's JIT
To summarize the above paragraph: instead of compiling to bytecode, compile to threaded gotos
(Lua is everything JS should have been.)
Lua also has coroutines and a few other things that don't exist in JS. JS certainly has the market share and libraries but there's quite a few areas where Lua does much better.
However, there are (were?) restrictions when C functions are present on the call stack between `coroutine.resume` and coroutine.yield`.
Edit: not that low level, actually, here's the coroutine implementation, built on top of the public Lua C API:
http://www.lua.org/source/5.3/lcorolib.c.html
(167 lines including white space and comments).
Two, core languages don't matter here because they are fundamentally similar to each other at the core level (dynamic and considerably weak typing, unified object-array, first-class value from missing indexing, semi-working lexical scopes and much more). The pain points of Lua tended to be same to those of JavaScript in my professional experience.
Three, the language is not a mere combination of syntax and semantics. Any evaluation should also account for user bases and ecosystem, and in my very humble opinion Lua spectacularly fails at both. I'm not going to assume the alternative reality---Lua has a sizable user base and its ecosystem is worse even for that user base.
I think that call depends on your use case. If you've ever tried embedding v8 or spidermonkey you might have a different opinion.
For instance we use to run the entire game state for multiple game titles I worked on in Lua. This was on the PSP where we only had a 400kb block of execution space and a 333MHz MIPS processor(system memory was 8mb w/ 24mb for video + audio).
Those types of environments are where Lua shines which lets you use a scripting language that scales from nothing up to full-blown systems.
I'm saying this in the embedded perspective (my professional involvement with Lua was primarily in that form). The lack of package manager might be acceptable as an embedded language---the lack of quality library isn't. The embedded story masks the weakness in the ecosystem because you can somehow make everything from scratch, but when you are programming at large the story becomes a disaster (combined with common symptom of dynamically and weakly typed languages).
I don't doubt Lua is a good language to embed. I doubt Lua is a good language to write a large software, especially after having worked with more than 300K lines of Lua code in a single code base.
The lack of quality library also means that you are even more risky when you are writing a small program (because you have less incentive to write it yourself). I have experienced multiple times that even the existing libraries (including the standard ones) had crucial flaws and no one seems to be bothered to fix that. Also in the embedded setting the use of snippets, rather than proper libraries, are more common as libraries can be harder to integrate, and unfortunately we are left with PHP-esque lua-users.org for Lua...
And there are so many languages with no such popularity boost that nevertheless have much better tooling than Lua. In fact, JS is very exceptional (in that the growth drove the tooling) and in many cases the tooling tends to keep up with the growth---Lua didn't, even though it could have done better. The alternative reality is an easy way to excuse this situation but also meaningless here.
> JS: 0.11499999999068677ms
> Lua: 0.0019999999999527ms
BUT, they don't print the same results.
Is there a way to debug it?
It should print the following string: "print(1 + 2)"
This is very cool.
I think there's some bug in indexing into Lua tables. print(str({1, 2})) should print "(1 2)", for example, but it triggers an integer overflow error.
It's actually capable of compiling to both Lua and JS, so I was going to show off a Lisp running on top of Lua on top of WASM which generates JavaScript, completing the circle of life.
More likely, native apps will port to WASM.
Let me know if you need more info.
EDIT: Ah, it's firefox-only.
Is there some way I can help get it running on Chrome?
Uncaught TypeError: WebAssembly.instantiate is not a function
at doNativeWasm (main.js:1)
at Object.Module.asm (main.js:1)
at main.js:1This usually happens when you haven't restarted Chrome for a while.
Chrome Version 57.0.2987.110 (64-bit)