Hacker News Reader

Comment by breuleux | Hacker News Reader

There would still be incentives for collusion (I "reproduce" your research, you "reproduce" mine), and researchers pretending to reproduce papers but actually not bothering (especially if they believe that the original research was done properly).

Ultimately, I'm not sure how to incentivize reproduction of research: it's very easy to fake a successful reproduction (you already know the results, and the original researcher will not challenge you), so you don't want to reward that too much. Whereas incentivizing failed reproductions might lead some scientists to sabotage their own reproduction efforts in ways that are subtle enough to have plausible deniability.

Proceeding by pairs is probably not enough. You probably need 5-6 replications per paper to make sure that at least one attempt is honest and competent, and make the others afraid to do the wrong thing and stand out.

View on HN

You could randomize replications a bit, take away the choice. Or make it so that if you replicated one group's result, you can't replicate them again next time. The key is a bit of distance, a bit of neutrality. Enough jitter to break up cliques.

I don't work in academia but in my experience professors are basically all intellectually arrogant and ego-driven, and would relish having time and space to beat each other at the brain game. A failed replication is their chance to be "the smarter guy in the room" and crack open some long-held belief. A successful replication would probably happen most of the time and be far more boring.

I could imagine, if such a thing were mandated and in place for a while, one could build her career on replications, as a prosecutor or defense. She would publish new research solely to convince her colleagues that she is sharp enough to play prosecutor or defense.

Anything has got to be better than what we have now, where apparently you can cheat and defraud your way through an entire decades-spanning career.

The tricky thing with randomizing is that science gets very specialized, both with equipment required and knowledge. So there may only be a handful of people whose work you can competently replicate.

And those same people are reviewing the papers you publish and will not hesitate to sabotage your career if you have made them look bad by failing to replicate their papers.

It is much much harder to sustain a conspiracy among many distributed people over time, than it is to fake your own research results.

Making fraud much less convenient will greatly reduce the amount of it.

So does increasing the penalties.

If you publish a paper with fraudulent data, methods, or results, and you received any state or federal funds for it, there should be prison time. You stole taxpayer money.

I'm not saying for when people are wrong, I'm saying for when you can prove someone knowingly lied. It won't catch anyone, and you need to bar to be high enough that people don't go to jail for being bad scientists, but right now there is zero social, professional, or legal risk is just lying your ass off to get the next grant and keep the spice flowing.

Nobody's going to do that when changing the numbers in your Excel sheet carries a risk of a decade or two in a minimum security prison.

Reply on news.ycombinator.com