Loopr: A Loop/Reduction Macro for Clojure
aphyr.com
aphyr.com
The OP also speaks of decreasing indentation as a goal instead of decreasing cyclomatic complexity. It is true that one tracks with the other, and I'll be the first to say that I too am an 80 character wide masochist, but optimizing code structure around decreasing indentation is dubious at best.
But when you get improved performance and decreased indentation, everyone wins :)
If you don't know Haskell the idea is simple, it is that the essence of MapReduce is for you to have an aggregating data type A with two operations,
empty(): A
combine(A, A): A
such that combine(combine(x,y), z) = combine(x, combine(y, z))
combine(empty(), x) = x
combine(x, empty()) = x
The requirement that `combine(x, y) = combine(y, x)` is not strictly necessary if the MapReduce implementation maintains shards and shard indices and is careful to only merge neighboring shards, the first property is then sufficient.To make this into MapReduce, you need your underlying data type X and an injection, (x: X) → A. This is the mapping part of MapReduce.
So for example “First10” is such an aggregation; given the first 10 (or fewer) in shard 1 and the first 10 (or fewer) in shard 2, you can form the first 10 (or fewer) in shards 1+2. The “map” function can use the empty value to provide “filtering,” so you get “the first 10 which satisfy this logical condition” directly.
Sum is another such aggregation, Count is a specialization of it, Mean is technically just a pair of Sum and Count. Oh, did I forget to mention? Given two aggregations you can easily produce an aggregation that does both. “give me the sum of this field and the first 10 records summed and the count of terms” ... One (parallelizable) pass for all three, because the aggregations compose.
So a macro API is possible where this is instead something like, (I don't know Clojure)
(loopM [x [1 2 3 4 5 6 7]]
(if (< x 5)
[(sum x) (count) (sum (* x x)) (first 3 x)]))
#=> [15, 5, 55, [1, 2, 3]]If it's a neat trick macro you use once every two months then you may never get used to it
But the fact that it yields a clear performance gain is a more objective "win".
It's not just stylistic at this point and I'm inclinded to give it a shot
https://reference.wolfram.com/language/ref/Reap.html https://reference.wolfram.com/language/ref/Sow.html
The fact that it's not open source is a huge deal breaker for me.
I can't deploy it on any interesting platforms, nor can I bundle the it easily to release to the public and guarantee that the public will be able to run the code.
I will just say I really miss the Jepsen blog posts. They were always fun to read and I learned a ton.
1. use vcvars64.bat, usually in ...\Community\Common7\Tools\vcvars64.bat or ...\Enterprise\VC\Auxiliary\Build\vcvars64.bat
2. lein uberjar
3. Setup GraalVM: set JAVA_HOME=C:\Program Files\GraalVM\graalvm-ce-java17-22.3.0
4. native-image --no-fallback -jar target\default+uberjar\clojuresque-0.1.0-SNAPSHOT-standalone.jar
(you have to tweak with the paths above, depending on your system.
The resulting executable isn't small, but it starts quickly and doesn't depend on anything on the system other than MS C-runtime redistributable.
That said, you can figure it out normally.
Using native dependencies will always be the hardest. I'd recommend first trying to use graalvm friendly libraries, and if not, libraries that are pure Java and don't have native dependencies.
For SQLite for example, you have to include the SQLite C driver, and that's where it gets a bit complicated.
See here for a demo build that includes SQLite: https://github.com/ericdallo/sqlite-graalvm-sample/tree/mast...
The most friendly way is to rely on this: https://github.com/clj-easy
It lists Clojure native image friendly libraries you can use. And it also includes some pre-configured dependencies you can depend on that will bring the correct build config.
Then just pretend like there are no other libraries yet for this "new" language. Or learn about the native image process more deeply and contribute to the effort to add easier support for more libraries.
Clojure like syntax, Rust like internals.
Alternative is to compile your Clj uberjar using GraalVM. Here's a project to document how to do that and which common libraries are compilable: