Hacker News Reader

Comment by gog | Hacker News Reader

Quite a few of these can be set in the HTTP headers as well, but people editing templates don't always have access or know how to do it.

BTW; not all UTF-8 text has a byte-order mark, I would say most of them don't.

View on HN

BOM is not valid in UTF-8 for one thing.

To be pedantic, the Unicode standard disrecommends the use of a BOM in UTF-8-encoded documents rather than declaring it invalid.

It makes zero sense to specify a byte order for an encoding in which it is irrelevant. It only persists because of a lazy vendor that can't encode Unicode correctly.

It would have been nice if every well-encoded Unicode document started with BOM and every legacy doc did not, instead of having to guess whether a doc is more likely UTF-8 or Latin-1.

Then concatenating to valid Unicode documents would no longer be valid Unicode. That is bad. And ASCII text would no longer be a valid UTF-8 encoded Unicode document. That is bad. And even when everything has finally switched to UTF-8 every tool ever will still need to handle the BOM. That is bad.

Guessing between valid UTF-8 and Latin-1 is only ever ambiguous when there are multiple non-ASCII characters in a row and all those sequences are made up of a lead byte with the correct number of trailing bytes. How often is that a problem for you in practice?

If you can't set the header, sure, put it in the head.

What I don't get is why so many say it is required in the head, even when you manage to set the header.

Reply on news.ycombinator.com