Supdate – A Clojure library for transforming nested data structures
vvvvalvalval.github.io
vvvvalvalval.github.io
To get a feel for the difference I translated a few of the examples to what they would look like in specter.
A few things I learned:
- supdate is very useful if the data you have is already regular and not too convoluted. By comparison, most of the time the data I pass to specter is a giant (>100MB serialized and compressed) rat's nest (data from every AWS service under the sun).
- supdate is very good at this "implicit projection/selection" (in the relational algebra sense). It feels a lot more like Datomic pull syntax that way than specter. By comparison, the equivalent specter thing made me write a pretty complex multi-path with submap to get the same behavior. By comparison, specter feels better at the logic programming equivalent of this: in specter, I express the data I want to address and specter figures out where it lives; in supdate, I tell you exactly where it lives, structurally.
- Implicit selection (per previous point) feels like something that's maybe most valuable for data representation, so, UIs? To quote Perlis, "It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures.", so something I have done a few times with specter is write a navigator for commonly accessed things (like the actions in an AWS IAM statement) and then reuse that all over the place. It's harder for me to see how to do that in supdate, but I guess it's all data so it can't be that hard :)
In conclusion: I think it does what it says on the tin. There are things that it can't express (or at least I can't figure out how), but it's very clear at expressing the things it can. I probably won't end up using it, but that's not an indictment of the tool as much as it tells you what we use specter for :-)
We do lose the embedded proofs that statically typed APIs give. We regain some of the confidence with clojure.spec'ing them.
I wonder if there's a way to make a compile time spec - enforce contracts at API boundaries, but let stuff like supdate work in a highly generic way in between. C++ templates and Rust traits could work, but everyone would need to agree on a core set to use.
I wrote a library that complements this nicely -- it's for transforming one deeply nested data structure into one that has a different shape, for example:
(f/transform {:a [1 2 3] :b [2 3 4] :c [5 6]} {k [vs]} {vs #{k}}) => {1 #{:a}, 2 #{:a, :b}, 3 #{:a, :b}, 4 #{:b}, 5 #{:c}, 6 #{:c}}
What skills does one need to have to be able to write such libraries for yourself? I'm just feeling like a mediocre copy&understand&adapt&paste programmer but rarely I'm able to create unique concepts/software.
The only difference is in the way I decided to treat arrays
(xxx {:a [1 2 3]}
{:a [_ _ inc]})
;; {:a [1 2 4]}
{:a {2 inc}}
Except
1. Much easier to grok syntax/easier learning curve
2. Not as powerful/generic
The cost of abstraction should not be ignored, and that's why a lib like supdate sometimes makes sense before reaching out to Specter, lenses etc. (although I love those too!)
(require '[clojure.string :as str])