Hacker News Reader

Comment by nik736 | Hacker News Reader

The weird thing about this model is that it spits out Chinese characters just like the chinese models when chatting with it in German. Happened to me quite often during the past few days.

View on HN

Opus 4.8 and Fable 5 did the same; they would output random Chinese characters and then say “I made a typo, let me fix it” and it’s just them removing that stay Chinese characters. Haven’t seen this since 5.1 family of models.

I saw it in earlier Claude models, but I haven't seen it for a least 7-8 months.

i’ve seen it and Cyrillic in the 5x family

Does this suggest Mistral is distilling Deepseek et. al.? Can distillation keep passing through with high fidelity, or is it lossy like copying a VHS tape?

It doesn't, even remotely. It's a typical failure mode in all models, some do this more often. It's not related to samplers either. Not necessarily Chinese characters either, they likely just occupy a large portion of vocabularies. For example Gemini 2.0 used to glitch in Telugu and Ukrainian, Claude 2.x and 3.0 used to switch between random languages for a few words when writing in non-English, once in a blue moon.

I interpret it to mean there is a lot of Chinese in the training corpus.

No. I've had pretty much every model I've worked with (list too long to recite) spit out the odd Chinese, Korean, Thai or other Asian chunk of letters from time to time. ChatGPT in various incarnations has suddenly switched to Chinese in mid-output and remained that way until told to switch back. The only one I haven't really seen this from as far as I can recall is Gemini, but then again, I haven't used it as much because it was performing quite poor for what I was doing.

Ha, ha. Just like when Tilly Norwood switched to Chinese while interviewing with Piers Morgan [1].

On a serious note, I remember getting spooked when I saw a bunch of Chinese characters dumped out into some of my server logs (pre-LLM era). When I pasted the Chinese content from my log into google translate, it was just complete nonsense. Later I found a bug that was dumping raw binary data into my log file which interpreted as Chinese characters when viewed in a text editor. Perhaps similar bugs existed in the models.

[1] https://www.youtube.com/watch?v=8VOfGcTqqq8

I've seen this quite a bit on bad quants as well. I ran an early and very lossy quant of one of the Gemma3 models, and it loved to just stick Chinese and Korean characters into the text, sometimes without seeming to realize it had mixed languages.

Can you please share: - what specific input you see this issue with? - which interface you used (API, playground, specific sdk version or a wrapper app)? - whether the chinese chars appear in the thinking section or the output?

Thanks.

Sorry it was not one time but it happened easily 5-10x during the past few days on different topics and prompts and I didn't store them. It felt like when pushing the model on some specifics in some niche topics in German (I have been prompting in German and the model responded in German). It was the acutal output and not the thinking traces and it was through the API. I jokingly asked it what the character meant and it didn't even make sense in the context so it seemed like random chinese characters.

happened to me with Gemini too, actually. it sometimes just randomly decides to answer in a different language.

What other languages?

Japanese or chinese, for example.

Google also does that surprisingly

you can see the same thing with any models

Havn't seen it for months with other frontier models.

Reply on news.ycombinator.com