Hacker News Reader

Comment by pilif | Hacker News Reader

> A utf8 decoder should reject portions of utf8 streams that don't use the shortest possible encoding

so you would say that there should be no file names using the cyrillic o? So if a russian-speaking person wants to save a file, that file name should be rejected? Or translated into a mish-mash between cyrillic and roman characters?

How will that work if that filename is reused on a system on which the default font doesn't contain the roman characters (I'm sure such a thing exists) and thus font substitution needs to happen?

The fix definitely isn't this easy. Maybe one could disallow homoglyphs of a different language than the one dominating the current file name. But this might be a lot of work and I doubt it's fool-proof.

View on HN

    > A utf8 decoder should reject portions of utf8
    > streams that don't use the shortest possible encoding

    so you would say that there should be no file
    names using the cyrillic o?

I'm sorry, I was unclear. I should have said "don't use the shortest possible encoding for a code point". Cyrillic 'o' is code point U+043E while roman 'o' is code point U+006F. The canonicalization attack relies on overly liberal utf8 decoders that would allow multiple binary streams to be interpreted as, say, code point U+006F.

This looks like the canonicalization attack, but is a different problem, one that is not solved by fixing decoders.

The Cyrillic o isn't being interpreted as U+006F, it just looks like it. A UTF-8 decoder doesn't know how it's output is going to be used.

That's not the point. The attack relies on some nonconforming decoders exhibiting a many-to-one mapping of bitstreams to codepoints, changing the semantics of the bistream.

you shouldn't restrict anything - just give the users enough hints to be more informed.

I would highlite the background of any character that is not from the users codepage in red.

for eg. if your local settings are us-en, any character not from that codepage will have a red background (or even in italics, some way to signify that the character is 'foreign')

A simpler fix would be to allow only ASCII characters in system folders.

So the "fix" will only work in system folders.

The "problem" described only exists in system folders.

Reply on news.ycombinator.com