Hacker News Reader

Comment by natch | Hacker News Reader

Strings (NSString) on Apple platforms are UTF-16. The Apple platforms are not exactly lagging behind in either multilingual, or text processing. I wonder what this team of three people knows that Apple doesn't? Or is it the other way around, that Apple knows something they don't, and when it comes to shipping products that work in the real world, Apple has figured out how to do it?

View on HN

NSStrings are opaque--you always call accessor functions and never have access to the low level backing store. The reason they are good is that you can't get data into or out of them without specifying an encoding, which leaves the actual encoding of the backing store as an implementation detail.

The fact is, I don't even know (or see documented) that the backing is UTF-16--Apple is free to change that at their whim and no user programs would break.

It's not documented (presumably) for that very reason.

In fact, the opposite is implied by initWithBytesNoCopy:length:encoding:freeWhenDone: - it should be possible right now to have NSStrings with arbitrary internal representations, even if most other creation methods currently convert to UTF16.

Unfortunately, this isn't entirely true.

  - (unichar)characterAtIndex:(NSUInteger)index

will return the UTF-16 "characters" (codepoints or half-codepoints)

NSString is decades old, from NeXTSTEP (you can see that in the name: "NS"). While its possible they could change it, the in-memory representation doesn't matter much in this case. When you transfer data out, such as with writeToFile:encoding: or convert it into a char * (often with UTF8String, or one of the C string methods), you are almost always specifying an encoding anyway. And, for most Cocoa apps, that encoding is UTF8.

As the authors explain in the post, many things use UTF-16 internally (Python, Java, C#, etc...). It does not mean that it's the best solution.

Have you read the article? They make a good explanation at why UTF-16 is "the worst of both worlds" (wide characters AND variable lenght).

Reply on news.ycombinator.com