Hacker News Reader

Comment by mike_hearn | Hacker News Reader

There are projects that try to do this, but you do have to use them.

LLVM is one. That's pretty successful.

The JVM is another. That's very successful, there are lots of languages that build on the JVM to reduce their tooling costs. Java, Kotlin and Clojure are examples of that.

GraalVM is another. If you use their Truffle framework then you get JIT compilation, debugging, tracing, instrumentation, different kinds of interpreters, advanced strings, some LSP support, language interop and FFIs and more.

But what you see, generally, language authors tend to start by hacking together a compiler and standard library starting from a C hello world program. Not that many survey the landscape and look for ways to reduce the costs.

View on HN

Racket, PyPy and Parrot (RIP) are also designed to host multiple languages. BEAM is another interesting VM (fault-tolerance, distributed systems, live updates, etc.). GNU has a bunch of projects to help language implementers, like Lightning, Jitter, libjit, superopt, etc. Eclipse's Xtext lives in the same space as tree-sitter and (to some extent) LSP.

Yeah part of the issue is that language designers by definition have not-invented-here syndrome. So it takes a lot to convince them to not write it themselves. LLVM has solved a sufficiently hard problem that it’s gotten over this barrier, but other meta-tools will likely struggle to achieve that

IMO, LLVM is more successful than JVM in this metric. JVM has definitely had some powerhouses, but it's inherent limitations has meant that it is only able to produce languages that have many of the same limitations as Java. Since LLVM generates machine code, the limitations of LLVM compiled languages end up being a lot closer to the limitations of modern processors.

JVMs also produce machine code ;) In practice LLVM is designed for C/C++ like languages. You can use it for other things but not easily. GraalVM is the closest thing to a universal VM so far, but even then, it's more optimized for high level languages than stuff like Rust (even though it can run those).

As Rust folks keep re-discovering in compiler regressions, likewise LLVM has many of the limitations of the C and C++ abstract machine model.

It seems like the main problem, for implementers of languages significantly different to C and C-like languages, is that LLVM doesn't offer direct access to hardware registers or the stack, and it has built-in calling conventions. (Although, that's kind of the point of LLVM IR. GHC implementers wanted certain values to be always kept in registers, from what I understood of it.)

This was a problem in writing an LLVM backend for both the Glasgow Haskell Compiler and the Standard ML of New Jersey Compiler. It seems like it was easier for the GHC implementers, because they were already able to compile their semantic IR (the spineless tagless G-machine) into C--/Cmm, which conceptually isn't that different from LLVM IR. The SML implementers compiled to the continuation passing style, which is harder to map to LLVM IR. However, the GHC implementers still had to extend LLVM with a custom calling convention, so it was still challenging.

Paper on the LLVM backend for GHC Haskell: https://llvm.org/pubs/2010-09-HASKELLSYM-LLVM-GHC.pdf

Paper on the LLVM backend for SML/NJ: https://people.cs.uchicago.edu/~jhr/papers/2020/ifl-smlnj-ll...

Thanks for the links.

At this point, between Rust, Julia, and Swift, it is in pretty good shape. The biggest one that used to be a problem was handling of undefined behavior, but at this point, LLVM's poison system works quite well.

Or does it allow you build a lot of very similar languages with slightly different looking syntax?

TruffleRuby is a good existance proof that it's not just that. (Clojure is another dynamic language, but it was designed to be hosted on the JVM so it's an easier case)

Clojure and Java are dissimilar in non-trivial ways.

Also there's a CL on the JVM - Armed Bear Common Lisp.

Though LLVM seems like it best maps the semantics of a language like C++, there's a fairly diverse lot of languages with LLVM backends, including Common Lisp, Haskell, Scala, and Swift.

Reply on news.ycombinator.com