Hacker News Reader

Comment by billpg | Hacker News Reader

Emojis are a plot to make English-speaking developers care about fixing their code to work with Unicode.

Not just Unicode, but Unicode characters outside of the Basic Multilingual Plane (BMP).

At the very least it is a pleasant unintended consequence. A decade ago support for characters outside of the BMP was hairy (certainly possibly though in PDF's with LaTeX via XeLaTeX), and something you would use if you dabbled in the obscure glyphs contained in it; now it is standard.

If only they hadn't tried to cut it down to 16 bits near the start, we could have avoided a lot of the partial support that emojis expose.

Maybe all Asian scripts was not planned to be included back then?

Seems strange they would miscount so grossly otherwise.

It's all down to CJK. Originally they allocated 21k codepoints to CJK, and if that was accurate then 16 bits would pretty much fit things.

But we currently have 88k CJK characters assigned out of possibly more than 100k total.

I can't easily find anything about how this went wrong and they got such a small number.

> we currently have 88k CJK characters assigned

If you also count the Unihan variations registered in Unicode's ideographic variation database by various Japanese outfits, encoded using the VS16 to VS255 characters after the codepoint they modify, there's another 8k or 9k unique characters assigned.

Ideally, we'd have started from UTF-8 and never even bothered with 16 bit chars in the first place.

As somebody not even working within the light-cone of this field, what is going on with emojis? It seems like everything I read about unicode has at least a mention of them. Are they that important to users? Particularly technically challenging? Interesting from a theoretical standpoint?

Reply on news.ycombinator.com