Comment by catch23 | Hacker News Reader

Comment by catch23 | Hacker News Reader

Having actually used MsgPack, it's nice that's compact, but bad that it doesn't handle utf-8 properly. The reason data is smaller is because they're basically using less bits depending the value, eg if the numerical value is "5" then you can use 3 bits to represent the value whereas JSON will always use floats to represent integer values. If you know exactly what your data in the JSON might be, MsgPack is nice, otherwise it can be a pain in the butt if you're sending arbitrary data from users.

Here's a nice review of different "competitive" formats to JSON: http://qconsf.com/dl/qcon-sanfran-2011/slides/SastryMalladi_...

According to their review, msgpack is good for small packets of data, bad for big ones. Binary JSON formats like MsgPack, are only good if you know your exact usage pattern, otherwise they bring along too many restrictions for it to be competitive. The best transport mechanism is still JSON.

"whereas JSON will always use floats to represent integer values."

It might use floats to represent integers in memory, but when json is transferred over the wire it's a textual encoding so each digit will consume 8 bits.

That's true, but I guess one should then mention a 5 digit number would consume less space in memory than in textual format. MsgPack recognizes this and stores that 5 digit number using the minimum number of bits required, but this wouldn't be possible in the usual json string formats.

What problems does it have with UTF-8?

It uses invalid UTF-8 chars all over the place, I guess it assumes text to be Latin-1.

‚¤¨£ - all those chars would be two bytes in utf-8, and even worse, you'll have to escape high utf-8 characters thus making your MsgPack message bigger than JSON if you're unlucky enough.

According to https://github.com/msgpack/msgpack/issues/26 it doesn't handle strings at all:

> This is by design. MessagePack doesn't have string type but only bytes type. So decoding bytes to string is application layer business.

So it looks like all of the claims of "just change one line of code" (and also most of these benchmarks) aren't quite true: unless you can guarantee your input text is ASCII, technically you have to first traverse your objects and replace all strings with decoded byte arrays. I would think that would be a dealbreaker for any user input.

Reply on news.ycombinator.com