Hacker News Reader

Comment by ubernostrum | Hacker News Reader

You can garble filenames just as easily in statically-typed languages. Consider, for example, Windows' infamous 16-bit units that aren't actually well-formed UTF-16. I'm not aware of any widely-used programming language whose type system will save you from that sort of thing ("here's some bytes, figure out if they're a string and if so what encoding" is a historically very difficult problem).

View on HN

Rust `String`s are always UTF-8 – distinct from `OsString`s which are not. Conversions are explicit, and the programmer is often forced to decide between conversions which are a) fallible or b) lossy. If neither choice is appropriate, the only option is to avoid the conversion, which… is correct.

https://doc.rust-lang.org/std/string/struct.String.html

https://doc.rust-lang.org/std/ffi/struct.OsString.html

https://doc.rust-lang.org/std/path/struct.PathBuf.html

> You can garble filenames just as easily in statically-typed languages.

If the language assumes filenames are regular language strings, which not all do.

> Consider, for example, Windows' infamous 16-bit units that aren't actually well-formed UTF-16.

unix filenames are literally just bags of bytes with no known or specified encoding.

Unix filenames don't pretend to be something they aren't. Windows filenames like to present a convincing façade of being UTF-16 right up until they aren't.

Windows filenames "like to present a convincing façade of being UTF-16" in the exact same way unix filenames "like to present a convincing façade of being UTF-8". Both are common assumptions neither is actually true, and all of that is well-documented.

I would express it more as "programmers in Unix environments like to act as if everything still uses the C locale everywhere, all the time".

> unix filenames "like to present a convincing façade of being UTF-8"

Except they never have? Unix paths have always been bags of bytes, both before Unicode and UTF-8 were invented and after. It's just convention that modern Unix systems use UTF-8 as the text encoding for paths.

> Except they never have?

And neither have Windows paths ever actually pretended to be UTF-16, that's my point.

Reply on news.ycombinator.com