Hacker News Reader

Comment by EvgeniyZh | Hacker News Reader

There are at least two problems with this argument. First, faking a research in a way that won't be detected is hard, especially if aim for high impact paper which probably will be studied by others. That's why we hear a lot about fraud in top tier venues. Lower ones probably have more problematic publications. Second, having fake results won't guarantee you a high quality publication, leave alone position in the future.

I can imagine some kind of first order phase transition in which the amount of fraudsters becomes so large that self correction breaks and instead they start to endorse each other. But that'd be evident from the outside, since science will stop producing real life results. We're definitely not there yet.

View on HN

> First, faking a research in a way that won't be detected is hard, especially if aim for high impact paper which probably will be studied by others.

Fake is a spectrum. Soft faking is easy and widespread--"p-hacking" is the polite term.

The stakes for faking it are low because most research is low impact, most research is seldom reproduced, and even bad research is rarely retracted. For that which is reproduced there is plausible deniability--"researcher degrees of freedom". Getting a high impact study published is more important than having one retracted, so researchers follow that incentive structure.

> But that'd be evident from the outside, since science will stop producing real life results. We're definitely not there yet.

Have you taken a look at the biosciences, social sciences, and medicine? Basic research hasn't been reliable for decades.

> The stakes for faking it are low because most research is low impact ... Getting a high impact study published

That's contradictory. For low impact studies you can fake whatever you want, because nobody really cares. That probably wouldn't promote your career either tho. Publishing bunch of organic semiconductor papers in Nature and Science will promote your career, but won't hold. That's the point.

> Have you taken a look at the biosciences, social sciences, and medicine? Basic research hasn't been reliable for decades.

Not deeply, but I surely can come up with bio/med research with real life applications that appeared in last 2-3 decades. Say mRNA vaccines?

The entire fields of sociology, social psychology, economics, literature, theological studies, etc are all low impact. Even if you get in the best journals, it’s high impact for your career but low impact to society so it will get very little scrutiny apart from the tiny slice of academia you compete with.

Again, I know that academia is very different among fields, but usually high impact for career ~ high impact for (sub)field, meaning people will want to look at your data and possibly try to replicate it. Outside scrutiny for science is extremely rare anyway

> That's contradictory.

What is contradictory in a claim that most research is low impact thus easier to fake while the incentives for publishing high impact research skews to publishing bad/fake papers since publishing is far more important then having a paper retracted?

> Not deeply, but I surely can come up with bio/med research with real life applications that appeared in last 2-3 decades. Say mRNA vaccines?

Nobody said that all research is faked.

Ok, not contradictory, it is just two claims unrelated both between themselves and to my original claim. Yes, low impact research is easy to fake. Yes, there is incentive to publish fake high-impact papers. Yes, it is hard to create and publish fake high-impact impact research.

It doesn't have to be outright fraud. I imagine the vast majority of manipulations are extremely subtle. Intentional selection bias, multiple hypothesis testing without correction or reporting the failed hypotheses(p hacking). Most of these would improve your chances of finding important results significantly and be almost impossible to detect

For the first argument I can't really speak beyond my field (physics, or specifically condensed matter physics), where I think it's fairly hard to be subtle. Often it's partial data presentation which allows for inverse Occam razor [1] (proposing fancy explaination instead of a simple one), which is why full data publication requirement are important (and seem to happen more).

In smaller sub (sub)fields sometimes fraud becomes large enough to self-sustain, like it happened with Majoranas [2,3] but even that kind of self-corrected. [1] https://arxiv.org/abs/2204.08284 [2] https://twitter.com/spinespresso/status/1503352928656138241 [3] https://espressospin.org/2022/11/17/the-fallen-angel-particl...

> without reporting the failed hypotheses(p hacking).

This one is so subtle, it's probable that many of the people doing it don't even realize they're doing something wrong.

It doesn't even need to be deliberately faked data but more data published without much due process to ensure it validity.

I have had about three people working in the research fields that have used the term, off the record of course, "Publish or perish".

Get it out the door, try to get the funding and hopefully if it works out, solve the issues before it becomes a problem.

Reply on news.ycombinator.com