Hacker News Reader

Comment by cbr | Hacker News Reader

This is not a canonicalization attack. Those attacks are based on there being multiple ways to encode the same unicode codepoint in utf8. A utf8 decoder should reject portions of utf8 streams that don't use the shortest possible encoding, but not all do. If there are multiple ways to encode '<', then an xss prevention filter is going to have trouble.

The attack described here is simpler: two unicode codepoints, roman 'o' and cyrillic 'o', usually look identical. So by substituting cyrillic we can make a file called 'hosts' that the operating system won't pay attention to. This is the same problem with punycode internationalized domain names, where paypal.com might be spelled with a cyrillic 'a' and mislead people. The fix for domain names was to restrict what unicode you could use where. I'm not sure what the fix is here, aside from always showing hidden files.

View on HN

> A utf8 decoder should reject portions of utf8 streams that don't use the shortest possible encoding

so you would say that there should be no file names using the cyrillic o? So if a russian-speaking person wants to save a file, that file name should be rejected? Or translated into a mish-mash between cyrillic and roman characters?

How will that work if that filename is reused on a system on which the default font doesn't contain the roman characters (I'm sure such a thing exists) and thus font substitution needs to happen?

The fix definitely isn't this easy. Maybe one could disallow homoglyphs of a different language than the one dominating the current file name. But this might be a lot of work and I doubt it's fool-proof.

    > A utf8 decoder should reject portions of utf8
    > streams that don't use the shortest possible encoding

    so you would say that there should be no file
    names using the cyrillic o?

I'm sorry, I was unclear. I should have said "don't use the shortest possible encoding for a code point". Cyrillic 'o' is code point U+043E while roman 'o' is code point U+006F. The canonicalization attack relies on overly liberal utf8 decoders that would allow multiple binary streams to be interpreted as, say, code point U+006F.

This looks like the canonicalization attack, but is a different problem, one that is not solved by fixing decoders.

The Cyrillic o isn't being interpreted as U+006F, it just looks like it. A UTF-8 decoder doesn't know how it's output is going to be used.

That's not the point. The attack relies on some nonconforming decoders exhibiting a many-to-one mapping of bitstreams to codepoints, changing the semantics of the bistream.

you shouldn't restrict anything - just give the users enough hints to be more informed.

I would highlite the background of any character that is not from the users codepage in red.

for eg. if your local settings are us-en, any character not from that codepage will have a red background (or even in italics, some way to signify that the character is 'foreign')

A simpler fix would be to allow only ASCII characters in system folders.

So the "fix" will only work in system folders.

The "problem" described only exists in system folders.

Reply on news.ycombinator.com