[flagged]
Demanding that everyone use our alphabet because we're too dumb to recognize the difference between google.com and xn--ggle-55da.com is the worst kind of "security through superior firepower"
(the fact that HN shows these correctly in the EDIT window but have to dump to punycode in display is sad, tbh)
Unicode is China forcing variable-length character encodings on the rest of the world.
A summary of a fascinating rabbit hole. Here are the scripts that are included in the UTF Supplementary Multilingual Plane, which cannot be encoded in UCS-2, excluding scripts of primarily historical and archeological interest. Obviously, they can all be encoded in UTF-16.
Deseret - The Mormon Alphabet, designed by Joseph Smith to remove corruption in English when written in the Latin alphabet. Used briefly in the 19th century, but no longer actively used, except by a small community of hobbyists and scholars.
Shavian - A phonetic alphabet for English, whose development was funded by George Bernard Shaw. Currently only used by hobbyists.
Osmanya - One of several competing alphabets for the Somali language. Officially superseded by the Somali Latin alphabet in 1972. "Material written in the script is 'almost non-existent'" -- Wikipedia
Adlam - Use to write the Fuldani language. Spoken by about 36m people. The Adlam script is currently taught in Fuldani schools, and is used widely.
Hanifi Rohingya - One of three scripts currently actively used to write Rohingya, the others being Rohingya Arabic, and Rohingya Latin. Rohingya is spoken by 1.5m people. There are at least two newspapers published in Hanifi Rohingya. It is often used in the Rohingya diaspora, and widely taught in Rohingya refugee camps.
Osage - Currently, the official alphabet of the Osage language. Invented in 2006. Osage is spoken by about 50,000 people, and children are taught to use the Osage alphabet in schools.
Medefaidrin - a constructed script and language created as a Christian sacred language by an Ibibio congregation in 1930s Nigeria. It has its roots in glossolalia ('speaking in tongues').
If you want to say that the facts are dumb, well, you can do that.
UTF-16 wastes a bunch of space (when you don't need 16 bits to mark a character), and doesn't avoid variable-width encoding. UTF-8 tries to optimize space (but is ASCII skewed) with more variance (but once you can't infer length from bytes, what's the difference?) UTF-32 is a space hog at the benefit of predictable length encoding (a full Int32 to represent the null char, or a carriage return)
It's the worst of both worlds!
Seriously nobody advocates UTF-16. The smart people advocate UCS-2.
> In particular, for the purposes of data exchange, UCS-2 and UTF-16 are identical formats.
https://web.archive.org/web/20030818043641/http://www.unicod...
If you don't like facts that is dumb.
Only 3-4 compilers stayed secure in this regard, by either disallowing unicode names, or by using TR39.
What else is a high security riscs? All names, such as path names or user names on OS kernels, like filesystems. Filesystem or user db's with unicode support but without identifier support such as TR39 can also lead to security attacks. Apple once had at least HPFS which did normalization, but the recently gave up, because nobody cared. And Linus still calls garbage in, garbage out valuable. Even if the famous ssh attack used such an homoglyph attack to stay undetected.
> Go to, let us go down, and there confound their language, that they may not understand one another's speech. So the LORD scattered them abroad from thence upon the face of all the earth: and they left off to build the city. Therefore is the name of it called Babel; because the LORD did there confound the language of all the earth: and from thence did the LORD scatter them abroad upon the face of all the earth.
Perhaps attempting to build one character set for all languages is just another Babel-like act of hubris, and the security vulnerabilities are God's judgment upon us for not sticking with the mess of incompatible local encodings that preceded it.
Having only ASCII in urls is incredible US centric. Yes it brings problems, yes they are solvable. It takes work to not have US and it's writing system to be the center of everything.