Hacker News Reader

Comment by 38 | Hacker News Reader

not self describing. if it was just the field names I could deal with that, but even the values are ambiguous, since the same type is used for Bytes and embedded Messages. the worst part is the wire type integer has two unused values, so they easily could have added a wire type for embedded messages

View on HN

Self-describing is point-less for serializations. There is a great deal of history here. ASN.1 has self-describing encoding rules such as BER/DER/CER, XER (XML), JER (JSON), and GSER (never mind), and it has non-self-describing serializations like PER (packed encoring rules) and OER (octet encoding rules). XML and JSON are self-describing, naturally. FastInfoSet is a PER-based fast encoding for XML, because it turns out that XML is slow (imagine that). XDR is a non-self-describing serialization format that resembles OER but with 4-octet alignment. Flat buffers is essentially an OER-ish encoding for the same IDL as protobufs, and is much better than protobufs.

It would be nice if the next serialization format either is truly original or just solves problems that somehow none of the many existing schemes do.

How many serialization formats are there? See: https://en.wikipedia.org/wiki/Comparison_of_data-serializati... (which is NOT a complete list).

> Self-describing is point-less for serializations

you couldn't be more wrong. what happens when you lose the schema, or never had access to it in the first place? think from the point of view of reverse engineering

Why would you lose it? NFS never lost its XDR schema, for example. Do you have any examples where the schema got lost?

I imagine when someone is choosing a serialization solution they probably don't really care about people trying to reverse engineer it... And if they did, they would just make schemas available instead.

And if you lose your own schema, then you probably have more serious underlying problems.

just imagine a world where protobuf replaces JSON, would you really want that? you're not thinking big picture

I actually don't think it'd be so bad.

Ideally it is capnproto instead of protobuf, but in general the same applies.

It is a _transport_ protocol. You can still write your config or data in any format you like.

you're not getting it. imagine if every single public website today using JSON switches to protobuf, you instantly lost most if not all ability to reverse those sites.

You are assuming websites offer an end user api via json but that's just a side effect, an implementation detail.

All these sites go through an evolution where they start out having open APIs, attracting third party developers, then eating their lunch and closing the APIs. No one in this sub-thread is saying that's desirable. But also nothing you can do can stop them from using something other than JSON.

All the binary JSON schemes, like CBOR for example, are all about making encoding and decoding faster. So the world kinda wants this even if you don't.

Do I want protobufs replacing JSON? No. I want flatbufs augmenting JSON (and XML, and ...).

I maintain an ASN.1 compiler and run-time that supports transliteration of DER to JSON. This is possible because the syntax/schema language/IDL and the encoding rules are separable. This is how you get the best of both worlds. You can use optimized binary encoding rules for interchange but convert to/from JSON/XML/whatever as needed for inspection.

Reply on news.ycombinator.com