Varlink is designed for IPC, so the expected data rate is maybe hundreds of messages per second in the worst case.
JSON can be parsed at 3 Gigabytes/second [0]. Even unoptimized stdlib parsers in scripting languages get 50 MB/sec.
That's more than enough, standard library support is much more important for a project like this.
And if there is ever a reason to send tar archive or video data, there is always "upgrade" option to switch to raw TCP sockets, with no JSON overhead at all.
Of all the things they could do, they decided to choose one of the worst possible formats for IPC data[1], maybe to pander to the web crowd.
[1] Maybe XML would be a close competitor.
I'm begging you to read the format it's replacing. https://dbus.freedesktop.org/doc/dbus-specification.html Ctrl+f Type System. JSON is beautiful, elegant, and minimal by comparison. yyyyuua(yv) It's like the printf message protocol but worse.
it isn't, fundamentally can't be as it has no support for binary blobs
and base64 encoded strings are a pretty terrible solution for this
and the space overhead of many small number can also be pretty bad
and while a lot of IPC use cases are so little performance sensitive that JSON is more then good enough and large blobs are normally not send over messages (instead you e.g. send a fd) there are still some use-cases where you use IPC to listen to system events and you have a situation where a pretty insane amount of them is emitted, in which case JSON might come back and bite you (less due to marshaling performance and more due to the space overhead which depending on the kind of events can easily be 33%+).
But what I do not know is how relevant such edge cases are.
Probably the biggest issue might be IPC system mainly used on 64bit systems not being able to properly represent all 64 bit integers .... but probably that is fine too.
For example, the metadata of the blob could be returned via JSON, then the request is “upgraded” to a pure binary pipe and the results read as-is.
binary blobs is just the biggest example and was only mentioned in relation to the "lingua franca" argument, many other common things are also larger in JSON. Only if you have many larger not escaped utf-8 strings does this overheads amortize. E.g. uuids are something not uncommonly send around and it's 17 bytes in msgpack as a bin value and 38 bytes in json (not inlcuding `:` and `,` ). That 211% the storage cost. Multiply it with something going on and producing endless amounts of events (e.g. some unmount/mount loop) and that difference can matter.
Through yes for most use cases this will never matter.
Yes, there’s a more optimal format than hex-encoding UUID values. However it simply does not matter for any use case this targets.
16 bytes vs 38 bytes is completely meaningless in the context of a local process sending a request to a local daemon. It’s meaningless when making a HTTP request as well, unfortunately.
I’d have loved Arrow to be the format chosen, but that’s not lowering the barrier to entry much.
sure most people don't write code for such edge cases, like ever
but e.g. systemd does sometimes because they reliable appear when running things at e.g. googles scale
True, but binary formats like CBOR or MessagePack are also that and efficient and have support for binary strings. This JSON is human readable thing is the nerverending argument of high level programmers. Anyone can read hex given enough time.
Peep the Python interface parser: https://github.com/varlink/python/blob/master/varlink/scanne.... This is the quality you can expect for an ad-hoc format, which is to say, pretty bad. Making the hardest part farming out to a parsing library you already have and is a built-in in your language is a no-brainer choice.
In it OP says it’s not slow, and the “hundreds of messages per second” you are referring to is not regarding the performance but the expected rate.
It’s possible for something to be fast, but only happen infrequently.