Dan Goodman – I will no longer do pre-publication peer review | Hacker News Reader

Dan Goodman – I will no longer do pre-publication peer review

twitter.com

8 pointskodisha5 comments

Hmm…. Removing peer review and self publishing turns his papers into blog entries. This has always been a concern about journals like PNAS where members can pay to publish - and their work was described as an ad. I guess this is helpful?

He is not arguing in favour of removing peer review but prepublication peer review. And I like the idea: in many disciplines it actually prevents finding error after publication, because the bar is really high to get any fixed after publication. We often write to authors because we cannot replicate results, many do not care after the managed to get published. Also there is the problem of publication bias: more positive results get published, so the whole system is often p-value hacking. IMHO very few conferences and journals have managed maintain any value in the editorial process.

Without pre-publication editing, you're commenting on a blog entry - he's the only editor. While there's a value in that, I would prefer the traditional literature approach rather than blogs. While I have not reviewed in the last few years, I was always a bother for authors that did not test properly or show an awareness of the state of the art in what they were talking about.

HN and other sites have proven that internet-based commentary isn't really of scientific use. More just entertainment.

You are assuming that the author is in control of the review process, why can't that be part of scientific organisation or e.g. conferences. Actually preprints already do this, the only difference is that things are typically finalized when things are published. Replication of particularly empirical research should be a center for a trust in results: it would incentivize authors to get more people to replicate your work. Also citations could rather serve as reviews instead of people blindly believing the peer review process and replicating often nonreplicatable truth.

You have to acknowledge that research is largely in a replication and publication bias crisis. Also it is flooded with nonfunctional journals with intransparent review processes. So it is good to think how to change this. Not saying that this is the solution but I like that someone is taking a stand.

I'm probably drawing parallels here that someone who has a better understanding of the academic world could validate or reject, but.. it feels a bit like there are more similarities emerging between published research and published software.

Citations are somewhat similar to dependencies, and post-publication review (as mentioned by the tweet's author) could be seen as similar to code review.

(note: when I say "more" similarities emerging, that's because I think there's already a positive trend towards publishing code and datasets alongside research papers, aiding reproducible results)

Reply on news.ycombinator.com