The Design and Implementation of the Wolfram Language Compiler (2020) [pdf] | Hacker News Reader

The Design and Implementation of the Wolfram Language Compiler (2020) [pdf]

dakkak.dev

145 pointsdaneelsan18 comments

Super cool to see a description of how Wolfram is compiled. I've used the Wolfram language for a long time but never really took a peak inside.

Some thoughts:

- The paper seems to suggest that compilation of scripting languages is not widely used outside academia. In fact, it's super common - all major browsers do it for JavaScript.

- What they call "soft failure mode" is what I would call "OSR exit" or what others would call "deoptimization". It's neat to know that Wolfram does this!

- I love that they designed a new IR rather than trying to reuse something else, or worse, trying to run optimizations over bytecode or AST. Also great that they go straight to SSA. The MExpr->WIR->TWIR->code pipeline is similar to what happens in JavaScriptCore's DFG compiler. We use the term "DFG IR" to refer to both the typed and untyped form, but it's exactly the same idea as what they're doing.

- They could use better benchmarks and more of them. Simple compilers do well on small benchmarks. Once workloads get very large and complex, you start to add even more crazy to the compiler. You learn a ton when your benchmark corpus goes to 1,000,000 LoC or beyond.

- The fact that there are thousands of tests makes me say "awww how cute". JavaScriptCore has close to 100,000 tests. But, thousands of tests is better than no tests, and of course different folks mean different things by "one test" - it's possible that one of their tests covers the equivalent of what 100s of our tests would cover.

Anyway, congratulations are in order - this is great work. It's always fun to see new compiler work for interesting languages, and Wolfram definitely qualifies as a particularly interesting language.

I wouldn't really classify "soft failure mode" as OSR; from my experimentation it reinvokes the entire function when there's a failure rather than migrating over the execution state.

In terms of tests, I get the impression that this compiler doesn't (yet) have to deal with much of the complexity a modern JavaScript one has to (OSR, GC info, inline caches, probably even most ISA-level details), so there are whole classes of tests that just don't need to exist.

Interesting that it just reinvokes the whole function. I always thought that Wolfram does have some effects - like, you can print things. You can’t rerun the whole function if you have done any effects. But maybe I’m wrong about the effectfulness of the language.

Wolfram may not have inline caches or GC but it has lots of super interesting things that JS doesn’t have. Must be a fun system to work on!

It has GC, e.g. DataStructures (https://reference.wolfram.com/language/guide/DataStructures....) are GC'd. IIRC it uses refcounting for this.

By GC info in my post I mean tracking which registers and stack slots at each safepoint contain object references so the tracing GC can precisely identify all of the GC roots.

Since Wolfram Language uses referencing counting instead, it doesn't need all of that complexity; it just has to insert MemoryAcquire/MemoryRelease at the appropriate spots in the IR.

I see what you mean, simpler indeed!

You don’t need to track registers and stack slots to do GC. JSC doesn’t.

The effects are how I tested this out; I used FunctionCompile on a function that uses first uses KernelFunction to call a user-defined function that increments a counter, and then calls Sqrt on the function's argument.

When you give it a non-negative argument, the counter increases by one; when you give it a negative argument you get the message of it reverting to the uncompiled evaluation and the counter increases by two.

Interesting...

Isn't this a potential for a lot of crazy bugs then?

I've never used Wolfram in earnest (touched it once in undergrad ever so briefly) so I may be misunderstanding.

EDIT: I see that the documentation for FunctionCompile indicates it's for pure functions only.https://reference.wolfram.com/language/ref/FunctionCompile.h...

In Wolfram Language, a "pure function" is what others would call an "anonymous function" or a "lambda". It doesn't actually imply the lack of side-effects.

The older Compile function also does the same thing when there's an error.

Given the difficulties of doing something fancier, I think it's a reasonable strategy. Mutating state that doesn't originate in a function isn't too common, and less so with the kinds of functions you'd compile. If the compiled function fails, you probably have some other bug in your code anyways.

3 authors, senior author is known not to code.

How many people worked on this thing?

On the paper or the interpreter? Based on the language first being released in 1988 I'd say quite a few for the second at least.

Curious - Any reference / citation for this?

Personal communication from 2012

Tom is the current main developer of the new Wolfram Compiler. Search in YouTube "wolfram compiler" and you'll see almost all the talks are his.

since this post might attract people that are mathematica powerusers: how good are mathematicas optimization routines vs commercial solvers like gurobi or cplex? reason i ask is i'm spinning up project that'll require a good bit of MIP, MILP and i have a mathematica license but i'm considering getting a gurobi license. since i've never tried gurobi i can't compare.

I never heard about Mathematica being used to solve large-scale MIPs. The best solvers in the market are very difficult to beat, since this is such a unique market with very specialized domain knowledge. So, while I guess it is possible to solve small to medium sized problems with Mathematica, I don't believe they're able to compete with some of the best solvers like Gurobi.

Reply on news.ycombinator.com