Hacker News Reader

I don't seem to get the efficiency table (or how efficiency is defined here?). Since Base65536 encodes 16 bits, why can't it encode UTF-16 with 100% efficiency? It says the efficiency is 64% instead.

I'm sure it's true, just curious why.

View on HN

Not every possible 16 bit value is itself valid UTF-16 - some need to be represented as two UTF-16 code-units, aka "surrogate pairs", 4 bytes total.

Not all of the code points used are (or can be) in the Basic Multilingual Plane. This means that when encoded in UTF-16, they come out to 32 bits, not 16. This skews the average number of output bits per input bit upwards.

Basic Multilingual Plane always make me think of Cthulhu... it lives beoynd

Reply on news.ycombinator.com