Hacker News Reader

The best thing a language (or library) can do is to not bless any single one of these as the default. Strings shouldn't have a length at all. They should provide properties/methods/accessors like byte_count, codepoint_count, grapheme_count etc. Make the user of the API think every time they're asking for a length of the string - which one do they actually need? Which one is the best for whatever they're trying to do?

View on HN

Can you name a single language that does that? I can't think of one.

None of the mainstream ones that I can think of. Which is really unfortunate, not the least because the defaults are all over the place - usually it's either bytes (when strings are UTF-8) or code units (when strings are UTF-16 - note, code units, not code points, so surrogate pairs count as 2!). Occasionally it's genuine code points, as in Python. Which, I think, goes to show why it's such a mess.

I think that if you treat strings as just lists[0] of UTF-8 code units, and code points, grapheme clusters, etc. are just views/adapters of those bytes, you're probably going to benefit the most.

[0]: When I say 'lists', I mean whatever the standard idea of a sequence of things is in the language. For C that's the array, or maybe the pointer+length pair. For Go it's a slice. For Rust, an iterator perhaps? For Python, it's a list.

Reply on news.ycombinator.com