Jaq – A jq clone focused on correctness, speed, and simplicity

With somewhat tabular data, you can use sqlite to read the data into tables and then work from there.

Example 10 from https://opensource.adobe.com/Spry/samples/data_region/JSONDa... (slightly fixed by removing the ellipsis) results in this interaction:

    sqlite> select json_extract(value, '$.id'), json_extract(value, '$.type') from json_each(readfile('test.json'), '$.items.item[0].batters.batter');
    1001|Regular
    1002|Chocolate
    1003|Blueberry
    1004|Devil's Food

    sqlite> select json_extract(value, '$.id'), json_extract(value, '$.type') from json_each(readfile('test.json'), '$.items.item[0].topping');
    5001|None
    5002|Glazed
    5005|Sugar
    5007|Powdered Sugar
    5006|Chocolate with Sprinkles
    5003|Chocolate
    5004|Maple

I personally don't understand why people aren't willing to learn instead. It's not hard to sit down and pick up a new skill and it's good to step out of one's comfort zone. I personally hate Powershell syntax, brevity is the soul of wit and PS could learn a thing or two from bash and "the linux way".

We seem obsessed with molding the machine to our individual preferences. Perhaps we should obsess over the opposite: molding our mind to think more like the machine. This keeps a lot of things simple, uncomplicated, and flexible.

Does a painter wish for paints that were more like how he wanted them to be? Sure, but at the end of the day he buys the same paint everyone else does and learns to work with his medium.

- https://steampipe.io/docs/sql/querying-json#querying-json #example w/the AWS steampipe plugin (I think this is a wrapper around the AWS go SDK)

- https://hub.steampipe.io/plugins/turbot/config #I think this lets you query random json files.

(edited to try to fix the bulleting)

    do more like the power shell way

    Everyone seems to want to invent their own new esoteric symbolic query language

[1] https://github.com/PowerShell/PowerShell

You could ask the same with respect to XML too -- why XPath/XSLT instead of SQL?

The problem is that SQL isn't that convenient when you're querying data in a free-form and recursive schema. Especially the latter, because recursive queries in SQL are just not pithy. I say this as someone who loves SQL.

[1] https://github.com/cube2222/octosql

N.B. those aliases are not created by default on *nix

It's pipeline-based and procedural, but you can be very declarative in data processing

Also re "lots of dependencies": This is kind of unavoidable in Rust because the stdlib is deliberately very lean, and focuses on basic data structures that are needed for interop (e.g. having common string types is important for different libraries to work together with each other) or not possible to implement without specific compiler support (e.g. marker traits or boxing). Contrast this with Go where the stdlib contains things like a full-fledged HTTP server and regex engine. It's easy to build things in Go with a rather short go.mod file, but only because the go.mod file does not show all the stdlib packages that you're using.

However what you say is good practice anyway. The spec (RFC 8259) has this note on interoperability:

> This specification allows implementations to set limits on the range and precision of numbers accepted. Since software that implements IEEE 754 binary64 (double precision) numbers [IEEE754] is generally available and widely used, good interoperability can be achieved by implementations that expect no more precision or range than these provide, in the sense that implementations will approximate JSON numbers within the expected precision. A JSON number such as 1E400 or 3.141592653589793238462643383279 may indicate potential interoperability problems, since it suggests that the software that created it expects receiving software to have greater capabilities for numeric magnitude and precision than is widely available.

Are you sure? Looking at https://www.json.org/json-en.html I don't see anything about 64 bit floats.

Many will produce higher precision but parse as float64 by default. But maximally-compatible JSON systems should always handle arbitrary precision.

Also, what!! Hey! Miss you man.

I think they kind of stuck in the development, even the mule engine only have one active developer from the github commit ….

It flattens the structure. And makes for easy diffing.

yq -o=props my-file.yaml

    (: file json2xml.xq :)
    declare default element namespace "http://www.w3.org/2005/xpath-functions";
    declare option saxon:output "method=text";
    declare variable $file as xs:string external;
    json-to-xml(unparsed-text($file))/<your xpath goes here>

    java -cp ~/Java/SaxonHE12-3J/saxon-he-12.3.jar net.sf.saxon.Query -q:json2xml.xq file='/path/to/file.json'

    for $price in json-to-xml(unparsed-text($file))/map/map/number[@key="price"]
    return $price+2

    {
      "fruit1": {
        "name": "apple",
        "color": "green",
        "price": 1.2
      },
      "fruit2": {
        "name": "pear",
        "color": "green",
        "price": 1.6
      }
    }

    <?xml version="1.0" encoding="UTF-8"?>
    <map xmlns="http://www.w3.org/2005/xpath-functions">
       <map key="fruit1">
          <string key="name">apple</string>
          <string key="color">green</string>
          <number key="price">1.2</number>
       </map>
       <map key="fruit2">
          <string key="name">pear</string>
          <string key="color">green</string>
          <number key="price">1.6</number>
       </map>
    </map>

But it's such a painful language to look at.

There's two classes of performance problems:

- implementation issues

- language issues

The latter is mainly a problem in `foreach` and also some missing ways to help programmers release references (via `$bindings`) that they no longer need.

The former is mostly a matter of doing a variety of bytecode interpreter improvements, and maybe doing more inlining, and maybe finding creative ways to reduce the number of branches.

The closest I’ve gotten is to wrap the APIs with GraphQL. This achieves joining, but requires strict typing and coding the schema+relationships ahead of time which restricts query flexibility for unforeseen edge cases.

Another is a workflow automation tool like n8n which isn’t as strict and is more user-friendly, but still isn’t very dynamic either.

Postman supports chaining, but in a static way with getting/setting env variables in pre/post request JS scripts.

Bash piping is another option, and seems like a more natural fit, but isn’t super reusable for data sources (e.g. with complex client/auth setup) and I’m not sure how well it would support batch requests.

It would be an interesting tool/language to build, but I figure there has to be a solution out there already.

    open http://… | select * where …
    # FROM can be omitted because you’re loading a pipe