Hacker News Reader

Comment by otabdeveloper4 | Hacker News Reader

I'd guess that 99.9% of those protocols are actually statically-typed.

E.g., an object's "foobar" field is always an array of int, and won't suddenly become a string from one invocation to the next.

It seems incredibly strange that we're somehow not leveraging this.

View on HN

What you're saying exists. Protobuf, flatbuffers, capnproto, xdr, parquet, arrow, avro, orc, sqlite. Indeed if you use these formats you will be able to load the data into memory more quickly than using JSON.

But this article isn't about those protocols. It's about JSON.

You can do it with JSON too. Various statically-typed languages do JSON parsing straight into structs all the time. (e.g. in the Rust ecosystem, Serde is the most popular implementation of this concept.)

Because it's relatively easy to make JSON-serialization fast if you know the shape ahead-of-time. It's a solved problem.

Here's a pure-JS solution: https://github.com/fastify/fast-json-stringify

> E.g., an object's "foobar" field is always an array of int, and won't suddenly become a string from one invocation to the next.

If only… I had to figure out how to deserialize a JSON API where a field could be: a) an object X, b) an array of X, or c) a string (mapping to X.Y)

That doesn’t need to be a problem. In Rust, the most popular thing in this space is Serde, with the serde_json crate. It can cope with such things just fine; define a data type that is capable of representing all the possibilities (perhaps as simple as an enum with a variant for each style, or perhaps it normalises them into one), and then… well, you probably have to write a visitor to help deserialise, which isn’t trivial, especially when you’re used to just deriving those things and so you’re not so familiar with Serde internals. But it’s possible. https://serde.rs/string-or-struct.html gives one example. The length of the example may be daunting, but it’s actually not too bad.

It was C#. I ended up writing a custom converter that converted all 3 cases into X[]

Reply on news.ycombinator.com