Hacker News Reader

Comment by TazeTSchnitzel | Hacker News Reader

I think this is a rare “useful” abuse of font ligatures, and in that respect I think this is a cool project, but it's also a terrible idea that nobody should actually use, because Japanese kanji readings are famously ambiguous and context-dependent. The modal character has two different readings without counting those used in personal names, nor contextual slight changes to pronunciation. Some characters have six different readings, and that's again without counting those used in personal names. There's no way a font can do a good job at it unless you can somehow stuff an entire LLM in there.

For context, it's a hard enough problem that Google Translate's own romanised pronunciation guides are frequently wrong, or disagree with their text-to-speech.

View on HN

> unless you can somehow stuff an entire LLM in there.

Well, it wouldn't be the first time ;)

https://fuglede.github.io/llama.ttf/

Was thinking of exactly this one as well.

Could we instead encode the kun or on reading in the kanji? It feels like something we should have placed inline in the encoding all along. That would literally fix this and you could build furigana rendering on top of that.

I agree, and if anything the problem is even worse. You mentioned characters with 6 readings, here's one with more than 30:

https://en.wiktionary.org/wiki/%E7%94%9F

And while you can usually figure out which reading applies based on what follows, there are still many cases where you can't. A standalone 今日 can be any of "konnichi", "kyou" or "konjitsu" depending on context and intent.

To be clear, that list contains about 12 non-name readings, of which about 7 are productive.

that's six more than can be displayed at once, and without looking at the entire sentence context it's not possible to choose the correct one.

Yeah. I also notice that Google Translate's guess at pronunciation frequently has errors, and that IS a LLM.

Someone further downthread said it appears to have about 80% accuracy on a short sentence, which means if you're not already at a level of skill where you can verify, then it's going to teach you the wrong pronunciations constantly. When studying languages myself, I find that learning the wrong pronunciation for something is annoyingly sticky, and unlearning takes more effort than learning something correctly the first time.

Oh yeah, there's a handful of words that I learned really early in Swedish that I always get the pronunciation of slightly wrong, even though I don't make that mistake in other words with the same patterns, because it's just so ingrained. And that's in a language with an alphabet!

> There's no way a font can do a good job at it unless you can somehow stuff an entire LLM in there.

I’d say you can do a pretty good job with conditional random fields.

Pretty good job, meaning that you get 96% there (if I recall correctly the results for MeCab which uses CRF). But 4% of kanji words is still multiple wrong furigana per page.

Still, very cool demo!

Yeah, I already see a few mistakes just looking at this page. But I guess still useful?

Yeah. I experimented with a random Japanese news article. Scanning it with my eyeballs, I found:

    Correct .. 80 chars
    Missing .. 12 chars
    Wrong ....  8 chars

So I guess the accuracy is roughly 80%. Maybe useful, but not super reliable.

> but it's also a terrible idea that nobody should actually use, because Japanese kanji readings are famously ambiguous and context-dependent

Seems a bit drastic. Don't use it as the main font in a japanese textbook, sure, but use it with the disclaimer that it's not perfect and will get some readings wrong? Why not, still pretty useful. Should people also not use Google Translate? (also, you can stuff an entire llm in a font, it's been done).

The point of furigana is to be exact - it's a pronunciation guide, and its purpose is to distinguish or to teach. If it's wrong, then nothing makes sense.

If I asked you 'are you a sewer (pronounced "soo-er")?', would that make sense? Maybe if you looked at it hard enough and thought about it, but in the meantime, I would probably get some very strange looks for asking.

It's funny to see LLMs being mentioned, because this is yet another case where "good enough" is, after all, not good enough once you know how "good enough" should be.

As far as I know, all the automated options out there (wether it's Furiganator, the chrome extension, or Google Translate's pronunciation) are far from 100% accurate. I still find both to be very useful, I just don't rely on them to be the absolute truth; they still help me on a daily basis. The web page for this font mentioned it didn't show any furigana if it was ambiguous (I haven't verified),and someone else here mentioned it was ~80% accurate. I respectfully disagree with the fact that it's useless if not perfect; I've personally found all these tools useful, and have not learned a single "wrong reading" because of them because I don't use them as a tool to learn kanji readings to begin with.

For language learning, I'd really recommend something like Yomitan here:

https://yomitan.wiki/

Yomitan only shows readings for words you explicitly hover over and press a modifier key, so it doesn't distract you, and when you do explicitly ask for a word, Yomitan will show all the readings and definitions.

I'd agree that 80% is better than nothing if you have no alternatives, but you do. I don't know about Furiganator, but Google Translate is also much better than 80%.

For that matter, furigana are commonly used to deliberately override ordinary readings. In some cases, it's more like kanji are used to explain the meaning of some katakana (especially, say, in-world jargon in a video game or LN) set as furigana above them. (Or some hidden or unusual connotation of a common word in hiragana.)

You're underestimating how wrong this is going to be on average. This is going to be nonsense like half the time. It's not comparable to Google Translate, it's more comparable to Babel Fish circa 1997.

> Why not, still pretty useful

Stumbling on names is fine, name litteraly have no logic, there is no system that will get them 100% correct by definition.

Screwing meaning is on a whole different level and Google Translate or even the built in screen reader of your OS will do an extremely good job given enough context (a whole phrase).

In real world use there is little space for sloppy hacks IMHO.

> Google Translate or even the built in screen reader of your OS will do an extremely good job given enough context

Google translate does not - I see it get readings wrong on very basic sentences with plenty of context on a daily basis. It doesn't make it useless. The website mentioned the font wouldn't show furigana if the reading was ambiguous so I was basing my judgement on that, obviously if it's just always wrong, that's not useful.

Reply on news.ycombinator.com