It also turns out that you very rarely need arbitrary string indexing. Most of the time, when you're indexing into a string, it's a fixed (and relatively small) number of bytes from the start or end. UTF-8 can do this in a tight inner loop that just checks for bytes that don't start with 0b10. If you need .startswith or .endswith, you can just compare bytes with a byte length offset. If you need to do substring search, you can do Boyer-Moore on bytes. If you need to test for equality, do it on bytes. If you need to chop a string in half for divide & conquer algorithms and only need to be approximately right, you can chop in half by bytes and then use UTF-8's self-synchronizing property to find the nearest character boundary.
[1]: http://erlang.org/pipermail/erlang-questions/2011-October/06...
Not all (base, combining+) have a precomposed version, so no that doesn't work.
* the first 14 characters were the length of the data stream in ASCII, padded with 0
* after this came the records - each record started with the size of the record, including the record header
* the record then had each field start with a letter that indicated what sort of record it was - integer, float, character, variable string - all were encoded in ASCII
* variable records were the letter "V", then a 14 byte length
Yes, the format was awful. But you asked when that would be useful. There's your answer.
This is why most modern programming & serialization languages (Go, Python 3, Protocol Buffers, Cap'n Proto) define separate byte[] and string types. Some things are just binary data and should be treated as such. Other things are encodings of world languages and should also be treated as such.