Hacker News Reader

Comment by CJefferson | Hacker News Reader

This test seems to have one major hole. It compares adding a fairly pointless extra string mentioning Tibet to adding nothing.

Maybe the model is getting confused. Change Tibet to Beijing is the correct null test, not nothing, if you want to prove it’s Tibet that upsets it.

View on HN

So the absolving theory is, maybe DeepSeek will get confused and mess up.... if you mention any place name?

Seems almost worse.

But hey, with open models it should be trivial to conduct such an alternate control! I'd be very interested in the results.

Yeah obviously that's the absolving theory - that's how a control works. Basic stuff. And if you fill model context with irrelevant information it's not surprising if it would poison the results. It could certainly be an alignment thing, or something malicious. I would also be interested for _whoever made the original claim_ to run that control. Because that's how science works. Because burden of proof is on who makes the claim. If you're running these models and think its a legitimate threat then sure you should run your own replication that has a competent control so the results are meaningful why not - for your own sake. Otherwise burden of proof is on whoever makes the claim. It that's you and it's trivial to conduct - go for it and let us know what you find!

All current models go off track the more you pollute their context with irrelevant information

Reply on news.ycombinator.com