Comment by anonymous908213 | Hacker News Reader

Comment by anonymous908213 | Hacker News Reader

Parent Full thread

anonymous908213

If there's real research that isn't methodogically handicapped and does a better job of reinforcing the claims you're making, it would behoove one to link it instead of "research" that selectively releases numbers to support a narrative.

> OP said “the political restrictions are applied at the presentation/UI layer only.” That doesn’t comport with the sort of error where model performance corresponds to specific geographic modifiers in a way other models do not.

It comports if the model performance has nothing to do with political restrictions. The selective numbers released do not establish that the degradation has any relevance to political restrictions; it could just as well be that introducing irrelevant information to context degrades model performance compared to not doing so. It could also be a result of political restrictions, but you'd have to have methodology that isn't completely useless to have a hope of establishing that.

> it would behoove one to link it instead of "research" that selectively releases numbers to support a narrative

This is like the endless “but in mice” comments.

Preliminary research has a place. We don’t conclude based off it. But we do validate questions and directions.

> it could just as well be that introducing irrelevant information to context degrades model performance compared to not doing so

Perhaps. You may like the rigor of the student paper, though it’s neither peer reviewed nor methodologically bulletproof and concludes ambiguously on whether this is corruption or a weird effect.

> We don’t conclude based off it.

You did conclude based off of it. You concluded that Deepseek is altering politically sensitive topics below the UI layer, and cited this 'research' as evidence of it.

It's not even accurate to call this research. As a researcher, you can't selectively release some numbers that support a claim while withholding other numbers that might contradict it. That's not science, just pure propaganda. Not only does it hold no value, it is actively harmful.

> You may like the rigor of the student paper, though it’s neither peer reviewed nor methodologically bulletproof and concludes ambiguously on whether this is corruption or a weird effect.

This does look more interesting, but doesn't lend itself to the strong claim originally presented, and peer review and strong methodology are indeed things you'd like to have before drawing conclusion based on research. It appears from their reporting that the alleged effect doesn't even exist in a statistically significant threshold. They averaged a -0.76% performance, far from Crowdstrike's headline claims that the Tibet prompt made the model 50% worse.

> You concluded that Deepseek is altering politically sensitive topics below the UI layer, and cited this 'research' as evidence of it

Where? I literally showed this report as a counter to the claim that “the political restrictions are applied at the presentation/UI layer only.”

> peer review and strong methodology are indeed things you'd like to have before drawing conclusion based on research

But not before raising suspicion.

> They averaged a -0.76% performance, far from Crowdstrike's headline claims that the Tibet prompt made the model 50% worse

Yup.

Reply on news.ycombinator.com