Hacker News Reader

Comment by geysersam | Hacker News Reader

If these are such well known techniques, why didn't the qwen people already use them to fix the bug in the model? Does the weight editing deteriorate the model in some other areas?

View on HN

Maybe they weren't aware of the bug, steering is a good method to quickly improve the model, more popularly used to refuse unsafe queries. Qwen researchers do work on interpretability and try to make it practical in production: https://arxiv.org/abs/2605.11887 , I've read this paper this summer. Overall yes, when you edit weights what often happens is that performance deteriorates elsewhere, this generally scales with how destructive your method of editing is and how expansive your edits are. Analogous to if you use a jackhammer and touch every surface in the house versus using a small chisel on a door frame.

My hope that in the future new methods will be better at editing larger scales without significant deterioration, maybe even performance increase instead.

They almost certainly knew...

https://huggingface.co/Qwen/Qwen3.5-0.8B#best-practices

> In thinking mode, we have observed that when using the recommended sampling parameters, Qwen3.5-0.8B is more prone to entering thinking loops compared to other Qwen3.5 models

There's also an active issue tracking this here: http://github.com/nobodywho-ooo/nobodywho/issues/557

Pretty sure that is also why this model was specifically chosen.

The entire process of AI training is weight editing. You're simply asking "Why didn't Qwen include more training cases to prevent looping?"

Reply on news.ycombinator.com