Hacker News Reader

> Thankfully, there’s an easy solution, which is normalizing the string into the “canonical form”.

Cool, problem solved!

> There are four standard normalization forms:

(╯°□°）╯︵ ┻━┻

I do understand the need for the difference between NFC and NFKC, but in hindsight NFD and NFKD seem entirely unnecessary.

NFD is useful if you want to do a diacritic-insensitive search.

The problem is that diacritic-insensitive search is locale-dependent, so it doesn't do the right thing anyway.

Specific example: in English, you'd want a search for ‘a’ to find ‘ä’ while this is the entirely wrong thing in Swedish where a and ä are distinct letters.

An English speaker probably wouldn't want a search for ‘i’ to match ‘j’ even though the latter just has an extra hook on the bottom.

That's a very good point. There's still a use for locale-insensitive diacritic-insensitive searches, but you're absolutely right that in most cases you'd want it to be locale-aware and therefore NFD isn't sufficient (though it may still be easier to do this on NFD than NFC).

I actually laughed out loud when I got to that sentence.

Glad that I still work in plain ASCII.

Reply on news.ycombinator.com