This article mentions Einstein being offended that his paper was submitted to peer review by Physical Review, and submitting it elsewhere instead:
"We (Mr. Rosen and I) had sent you our manuscript for publication and had not authorised you to show it to specialists before it is printed. I see no reason to address the – in any case erroneous – comments of your anonymous expert. On the basis of this incident I prefer to publish the paper elsewhere."
I just recently ended up there via this blog: https://experimentalhistory.substack.com/p/the-rise-and-fall...
which argues, not unconvincingly IMO, that peer review is a failed experiment which tries to stop bad science at the price of also stopping brilliant, unconventional science, where instead we would profit more from letting the scientific public do the sorting instead of hand-picked specialists. The author phrases it as science being as strong as its strongest links, not as weak as the weakest one, and we should do everything to get the brilliant crazy ideas.
Also currently on the front page: https://news.ycombinator.com/item?id=34257150
In brief, London 'man off the street' solves 20k year Ice Age drawings mystery, published in Cambridge Archaeological Journal via contact with Durham professor. (And he is first author.)
I don't say that to disagree with you by the way - it's undoubtedly a rare example.
I tend to agree that peer review has flaws. But given the ease with which someone can publish anything with no review whatsoever it seems reasonable that peer review might shift to fill a niche that is slower and less disruptive.
Most recent example: https://www.quantamagazine.org/long-out-of-math-an-ai-progra...
Rather, journals promote well-connected kooks at the expense of less well-connected kooks and happen to occasionally publish some science as a byproduct.
So "peer review" isn't holding back kook theories at all on its own. At best, journal prestige is bolstered by it, which the journal can then translate into holding them back. The track record is a bit unclear, though.
Journal aren't just a magic upload into scientific consciousness; it takes time and energy to read and process them.
And we nevertheless pay the price of adhering closer to orthodoxy for it, since effort being put in is a rather superficial "this paper sounds reasonable to me" filter, perhaps with some boxes that must be checked like p-value or sample size.
The problem is that inherently peer review is designed to act as a brake: >Peer review, in which a scientist's paper is scrutinized by a group of colleagues, is designed to find errors and weed out half-baked ideas.
But in blocking you are going to reject ideas. >And although some have criticized peer review for rejecting new ideas just because they're too radical for the establishment to stomach
How much of a problem and how many revolutionary ideas are stopped, I don't know. I am more concerned about the reproducibility problem in science.
Peer review is also obviously a bad solution for the reproducibility problem, given that we've got the problem now, concurrent with generations of peer reviewed manuscripts.