An alternative that completely replaced jq for me: rq https://github.com/dflemstr/rq
Not limited to JSON, and super fast.
Not limited to JSON, and super fast.
~> curl -s http://httpbin.org/ip | rq
{
"origin": "184.100.112.11"
}
~> curl -s http://httpbin.org/ip | rq 'at origin'
"184.100.112.11"
~> curl -s http://httpbin.org/ip | rq 'at origin | split "." | spread | drop 3'
"11"
~> curl -s http://httpbin.org/ip | rq -C
?foriginn184.100.112.11⏎
The only thing I haven't figured out is how to unquote string elements once filtered down to just that string.
Feel free to submit a feature request about the raw string use-case, I've been thinking about it for a while but some people have wanted me to add line support (maybe -l and -L?) and some people have wanted raw text support (maybe -r and -R?) while other people still have been talking about TSV/CSV support (not sure how that would look tbh...). I'm not sure what to do exactly, so feel free to tell me!
http://www.compciv.org/recipes/data/touring-the-spotify-api/
(just noticed that you used the Spotify API too for the demo :) https://github.com/dflemstr/rq/blob/master/doc/demo.md)
I personally like using jq but not sure how easy-to-grok it is for people who are new to both the command-line and to serialized data structures in general. Will have give rq a spin (waiting for it to compile on my machine) but it looks quite nice, at least for my own uses, which heavily involve YAML.
In terms of TSV/CSV support, you may want to checkout csvjson, which is part of the csvkit command-line suite: http://csvkit.readthedocs.io/en/0.9.1/scripts/csvjson.html
csvkit is my go-to tool for data, so much that I rarely ever use SQL for anything other than when I actually need a database. I suppose if rq were to be my go-to data-processing tool, I'd probably do this:
csvjson mydata.csv | rq ...
curl -s 'https://api.spotify.com/v1/search?q=rihanna&type=track&limit=50' \
| rq 'at "tracks.items" | spread | flatMap artists' \
| in2csv -f ndjson
Quick check whether it is super fast for a simple task like extracting a single field from 1M JSON objects:
$ time rq -j 'at "title"' < fixtures/1m.ldj > /dev/null real 2m16.692s
$ time jq .title < fixtures/1m.ldj > /dev/null real 0m26.040s
So jq is 8 times faster? Anyway, rq functions look useful, so will definitely check it out.
That v8 crate looks interesting. There was a post a while back about Neon, for writing Node modules in Rust: