Hacker News Reader

Comment by nextos | Hacker News Reader

You are totally right about peer review being misunderstood as replication. That's why preprints have gained so much traction lately. Things have become so complex that sometimes peer review is only good enough to filter results that are obviously wrong.

But replicating a proof or a complex experiment is way beyond what reviewers can accomplish.

Another problem with peer review, especially in experimental fields, are malicious reviewers. Big PIs typically gain control of a particular subfield by publishing something first, then getting lots of papers to review and blocking said papers and/or stealing good ideas from them.

View on HN

Big PIs typically gain control of a particular subfield by publishing something first, then getting lots of papers to review and blocking said papers and/or stealing good ideas from them.

That's a bold claim, especially the "typically" part. How can you back it up?

I know it's a bold claim. But PIs that dominate a field employ predatory tactics that are very similar to those used by big corporations. It's just economics.

I have witnessed this in some groups I have been a member of. Very unethical behaviors. As I said above blocking papers in peer review, or even rejecting them, and simultaneously sharing key bits from said papers with postdocs is a routine practice.

Knowing journal editors and reaching publication agreements before papers are even written is also very common, and hardly surprising to someone that has been in the field for some time.

I have absolutely seen this in my field. The top-of-its-field journal editor actually requested our paper for publication and one of the reviewers poured vinegar all over it, creating requirements literally no paper of this kind had ever been held to. We actually sent it to another journal, and got the verbatim same review back. They had sent it to the same top reviewer. Amazingly, said reviewer comes out with a pre-print of a similar paper that addresses some of his own concerns, about 9 months later. Amazing how that happened, hmmm?

The one consolation is that at least we know who's reviewing, who's an asshole, and that this space is as hot as we thought it was.

That sucks. I guess the solution is to publish preprints and move quickly. Then ignore scammy reviewers and publish elsewhere.

Proofs aren't very hard to review, especially because math doesn't have the proprietary problems of wet science so drafts aren't hidden. Such is the benefit of working in a field no one can profit from :-) Math doesn't have a replication crisis; false papers are rare.

Proofs are actually incredibly hard to review. Problems will be found in what had been accepted to be a "good" proof years to decades later. There's a whole movement to move proofs over to something/anything more verifiable (e.g. representing all proofs in Coq--but even then you're relying on the Coq proof assistant to have zero bugs).

Furthermore, the standard for mathematical proof has also changed over time, most significantly in the early 20th century. This led to a number of existing results needing to be re-proved (or thrown out! Some were incorrect!).

Exactly what qualifies as a proof is a FASCINATING debate. Mathematics is created by consensus, just like all other knowledge.

What are some examples of proofs that were originally accepted and then later shown to be incorrect?

Here are a few of them:

https://mathoverflow.net/questions/35468/widely-accepted-mat...

Proving a math problem is mentally challenging, but there are other interesting definitions of hard. In medicine, for instance. None of it requires particularly fancy logic. But can you cite any instances in math here you had to invest, say, $10-15M and collect data from every available human patient at multiple hospitals over 2-3 years in order to replicate?

Reply on news.ycombinator.com