Hacker News Reader

Comment by mardifoufs | Hacker News Reader

What's the alternative to json?

Most binary JSON variants including MessagePack deviate from the spirit of JSON in one fundamental way: they are not simple. The JSON file format can accurately be described with a few simple syntax diagram that fit comfortably on a printed page and the spec can be implmented with very little code. A binary file format comes with the theoretical further simplicity advantage that you can leave out conversion of numbers, detection of string length, (un)escaping and dealing with white space/pretty printing. Instead of taking that as gains and be done with it, such formats start introducing (space) microoptimizations, such as many different number formats, variable length numbers for sizes, a wealth of additional data formats such as dates, etc.

I think there a two good variants of binary JSON equivalents: one where you have a minimal implementation that makes both reading and writing with the lowest amount of code possible, potentially feeding a SAX-like interface or constructing an in memory representation (DOM-style).

The other one would be to facilitate lookups in large documents such as json_extract without parsing the whole document. That would require some clever layout of the data, potentially even as in-file hash tables or similar. I assume that is what SQLite has done.

TBF&H, when you say additional data formats such as dates, dates (including time) are the one format that would be great to have standardized in JSON, due to their inherent complexity. A format for binary data is also missing, but that is downright trivial to do ad-hoc (string literals + encoding hint) when compared to datetimes.

I don't know, it is trivially simple to choose either Unix timestamp (which is relatively close to what MessagePack is doing) or JSs format [1] if you like a bit more readability and time zone deltas. Or any of the zillion more complex formats such as ISO 8601, etc. In my opinion that would be part of the schema and not the file format.

[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

I wrote what I think may be the simplest possible serialization format, though I don't expect it to be a standard anytime soon

https://github.com/civboot/zoa

that is one of the more complex serialization formats I have seen.. custom backslash-escapes, 13 operators, "variable compiler", string concatenation, user-defined types (somehow only in binary protocol), two types of maps, special "file path" type...

I could say many things about it, but "simplest possible" is not one of them.

I misspoke, I should have said protocol. I'm referring to the bytecode not the text format. I'm probably throwing away the text format, for the reasons you outlined

As I said, I mispoke. Here's how I think a JSON equivalent format should be structured: https://github.com/civboot/zoa/blob/main/notes/dynamic.md

"Simple" in that it's trivial to decode with a bytestream and a jump table, and it can be written down very briefly.

You inspired me to re-examine it. I wrote up the new design here:

https://github.com/civboot/civlua/blob/main/zoa/README.md

It's still in it's initial stages, but I'd love feedback :D

There's tons of alternatives. BSON, CBOR, and protobuf are three arbitrary ones.

XML (:

On the database?

There's actually quite a lot of XML/XSLT support in various proprietary databases (eg RDF databases), and there's Postgres support too:

https://www.postgresql.org/docs/current/functions-xml.html

Iirc XML support in Postgres predates JSON support.

Woah! Ok I didn't know that. How's the performance?

Very XML'ish

Reply on news.ycombinator.com