Babashka: Fast native Clojure scripting runtime | Hacker News Reader

Babashka: Fast native Clojure scripting runtime

babashka.org

204 pointszbentley24 comments

This is a particularly joyful piece of open source software.

And I love this page describing it. I hadn't seen it before today.

After seeing Michiel Brokent -- the project's original creator/author -- present on it at a meetup, I wrote a tidy overview of babashka (bb) which you can find here: https://amontalenti.com/2020/07/11/babashka

Michiel Borkent, aka borkdude (transpose the r and o).

Ah, indeed. My mistake mis-spelling his last name.

Babashka has unlocked so much power for me. Clojure and the JVM simply aren't set up for rapid start up, Babashka is instant. Bash scripting is alien, ugly, and hazard prone for me - Babashka makes it totally familiar, since it's virtually identical to Clojure. That is the definition of "have my cake and eat it too".

The end result is that anything the least bit non-trivial becomes a Babashka script, and I've created tools containing dozens (or potentially, hundreds) of sub-commands.

Joyful is the correct adjective.

Babashka Babooka: Write Command-Line Clojure - https://news.ycombinator.com/item?id=34261010 - Jan 2023 (14 comments)

Babashka is a fast-starting scripting environment for Clojure - https://news.ycombinator.com/item?id=33909241 - Dec 2022 (42 comments)

Babashka Clojure nREPL as a system interface - https://news.ycombinator.com/item?id=33761877 - Nov 2022 (23 comments)

Clojure Scripting on Node.js - https://news.ycombinator.com/item?id=32339979 - Aug 2022 (33 comments)

Ad-hoc ClojureScript scripting of Mac applications - https://news.ycombinator.com/item?id=29779353 - Jan 2022 (8 comments)

Replacing my Octopress blog with 200 lines of Babashka - https://news.ycombinator.com/item?id=28982952 - Oct 2021 (27 comments)

Clojure Babashka scripting notebook - https://news.ycombinator.com/item?id=25227419 - Nov 2020 (2 comments)

Using Clojure in the command line with Babashka - https://news.ycombinator.com/item?id=24353476 - Sept 2020 (36 comments)

Babashka: A quick example - https://news.ycombinator.com/item?id=22382381 - Feb 2020 (16 comments)

Babashka – A Clojure for the grey areas of Bash - https://news.ycombinator.com/item?id=22254793 - Feb 2020 (65 comments)

Babashka: a tiny Clojure in Clojure for shell scripting - https://news.ycombinator.com/item?id=20667509 - Aug 2019 (1 comment)

Babashka Babooka is a very well written tutorial. I can't recommend it highly enough.

Sorry if this is a stupid question, but I see comments like this with previous posts on a big number of hackernews posts. Are these auto-generated and what is the point of them?

Not automated but machine-augmented: https://news.ycombinator.com/item?id=35668525

If someone took the time to implement Clojure as a Truffle language [1] it could also be JITted essentially for free (Truffle automatically generates a JIT from an interpreter implementation; further information can make the JITted code more efficient).

There seems to have been an academic attempt to do that (for a subset of the language) with promising results: https://epub.jku.at/obvulihs/download/pdf/501665

[1]: https://www.graalvm.org/latest/graalvm-as-a-platform/languag...

I suspect you could also get a long way even without Truffle. Last time I looked at Clojure startup it was doing a lot of really inefficient stuff that would be optimizable with modern JVM features. For example most of Clojure's startup time goes into giant <clinit> class initializers that init endless sets of constants for the entire standard library, regardless of what's actually used. Newer bytecode features like constant dynamic, invokedynamic etc would be ideal for this stuff but the Clojure compiler doesn't use them.

You can combine technologies. Compile the Clojure compiler to a native image as a library to make text->bytecode fast, then invoke it from the babashka wrapper if there's no JAR for the script inside a disk cache, otherwise run the JAR if found. If the Clojure bytecode were lazier and you combined it with AppCDS the startup time would be fine.

I've got a similar sort of concept but using Kotlin. The scripting syntax is convenient for the use case of replacing bash and the generated bytecode is simple and efficient. Unfortunately compiling Kotlin is really slow, so whilst finished scripts start up nice and fast the edit/compile/run cycle is poor.

Clojure's startup time on OpenJDK could certainly be improved considerably with Leyden with or without a Truffle implementation, and it is also true that Truffle is somewhat orthogonal — you could have Truffle Clojure running on OpenJDK or on Native Image.

However, if you want to use Native Image, then I think Truffle is the only viable way to improve the runtime performance. Even if you NI the current Clojure compiler, there would still be no way to run it short of running the resulting classes on OpenJDK, and those classes would need to be intialised, so NI would buy you little. I don't think that loading and initialising the compiler is the dominant factor in Clojure's startup; I recall it being the loading of user/library classes.

Right, so in order of rollout difficulty for the end developer:

1. Make the Clojure compiler emit more efficient and lazier bytecode, so it doesn't init everything all at once at startup (the stdlib gets compiled into what is effectively one mega-class). No change required to anything, it just starts up faster once they upgrade to a new Clojure version. Constant Dynamic was made for this problem.

2. AppCDS, Leyden. No semantics change, only workflow.

3. A Truffle version that operates on source code. Some workflow changes. Have to switch to a new runtime, eliminate build steps and ship sourcecode instead of bytecode (which may not matter).

4. Compile end user programs themselves down to native with native-image. Semantics and workflow both change but all the dead code in the stdlib can be eliminated this way.

Lots of options. (1) is the lowest hanging fruit but I suspect the intersection of people who know about the newest bytecode features, people who can modify the Clojure compiler, and people with the time and energy to do so, is currently the empty set.

I agree, just note that a Truffle implementation doesn't require a new runtime (if by runtime you mean a Java runtime; Truffle works on OpenJDK) and that the implementation of Babashka already depends on a Clojure interpreter (just not one based on the Truffle framework) that's a different implementation from mainline Clojure (which uses a compiler to bytecode).

My point wasn't so much to suggest to Clojure's stewards a course of action but to say that since Babashka already makes use of Native Image to get fast startup and already depends on a separate interpreter implementation that suffers from runtime performance, then Babashka may want to consider basing their interpreter on Truffle to improve runtime performance. If you already picked Native Image for your product and already decided to create a separate implementation of the language based on a source-code interpreter, might as well try basing it on Truffle.

That's true. Adapting or rewriting their interpreter for Truffle would indeed solve these problems too.

One of the side projects I keep wanting to try is to see how far I can drive GPT-4 to develop a Lisp interpreter for Truffle. Lisps are simple languages, and it does know at least some of the API.

Interesting; I've been using GraalVM to compile my Clojure JAR files for command line tools. Doing a basic Unix `time` command, the startup time for that is about 20 milliseconds, well within my tolerance threshold of a command line application.

That said, the executables for GraalVM are enormous, so I'm absolutely open to something comparable that avoids that. I'll give Babashka an install and play with it.

Babashka is built using GraalVM. It allows you to write script-like programs in Clojure while having instant startup time.

I like Clojure, but I never had any good opportunities to use it other than for a few small hobby projects. It is unfortunate that it is so huge with tons of dependencies and no simpler native implementation. I started looking at various LISPs and Schemes to find something lighter to use instead and ended up settling for Janet that I think is Clojure-like enough to be comfortable to use, but in a small native binary with no dependencies and can be embedded in other native programs. I am sure for big, real, projects that Clojure makes more sense, but for my hobby projects and scripts I do not think I will install it again. I am still happy for the things I learned from learning Clojure. It was a real eye-opener for an old OO-programmer.

https://janet-lang.org/

My initial experience as a low-time Clojure programmer* is that the language feels rather small and tidy (in a good way). Not as small as Scheme, but still very easy to learn the entire syntax and the native data types.

* - More than half of my time with Clojure has been with Advent of Code challenges the last 3 years, so I'm in the ~250 hour club.

Is anyone using Clojurescript for server-side Clojure with Node/Deno/Bun? I'm not even sure this is possible, but it seems to me like the most obvious candidate for a lighter-weight Clojure environment.

My question with Babashka is always: why? Why force Clojure into something it is not? Why force some interpreter through the heavy GraalVM machinery to have a binary interpreter for a language that wasn't meant for this? Just because "the syntax feels similar"? Might as well just use Scheme for such tasks, and invest the resources into other Clojure projects - Clojure has a need for that.

Nonetheless it is an impressive achievement. Just one that I deem, like GraalVM's native compilation itself, an unnecessary one.

You seem to suggest there is some platonic ideal use of clojure and this isn't it. But this simply isn't the case. Just because the first version of clojure was on the JVM doesn't mean all other uses are somehow 'lesser'.

Why is simple, use the language you already know that has good libraries available for scripting.

The thing is that this - unlike ClojureJVM, ClojureScript, ClojureCLR - only acts as if it were Clojure, while not actually being it, by lacking some core features and having a out of technical necessity convoluted "pod" mechanism.

We're using it at work, together with Clojure, it works just fine, but it is the weird sibling.

I see it differently. Consider a dev organization who is considering adopting Clojure as a supported (or even default) language. The case for such adoption is strengthened if you can make the argument that much of the shell scripting tasks can be done in the same* language and in a way that's more ergonomic and safer.

That helps drive clojure adoption in a way that suggesting that instead dev teams adopting clojure should also learn scheme and write some of their code in that seems not nearly as compelling.

It's the same reason that clojurescript is supportive of the overall clojure community, rather than being a distraction and diversion of resources.

* It's not 100% the same, but for what's needed to replace bash scripts, it rounds to 100%.

It allows people who know clojure use it for scripting without the load time of JVM. Most similar implementations of other languages is to overcome some limitation of the original implementation. Like Pypy for python. People do not have to invest time to learn a different language for every use case

Reply on news.ycombinator.com