Hacker News Reader

Comment by Retric | Hacker News Reader

A reasonable compromise IMO is for Journals to have a very temporary lock on new publications. Public access after say 1 to 2 months seems to meet the vast majority of "Open Science" requirements without much change required.

2) IMO a much more stringent form of peer review is necessary. P<0.05 is only useful in very limited situations. I would argue that unless you republish your exact experimental design beforehand you would need to drop that to P<.01 to cover for a lot of marginal research out there. The old do 10 studies and you can likely publish one of them is harder when you need to do 50 or fudge the numbers harder.

View on HN

The peer-review before an article is published is just a sanity check, like Fizz-Buzz during an interview.

The more valuable check is what happens after a paper is published, when your peers read the article and discuss it and critique it and try to replicate it and publish responses to it. What makes peer-reviewed journals valuable to science is the that lots of practitioners read it; the value of peer-review is that it makes the journals more interesting for practitioners to read because it doesn't waste their time on obvious crap.

Publishing a paper is not a stamp of truthiness. It's a communication to the community that something is interesting. Maybe it's right, maybe it's wrong, time will tell. Truth emerges from a large body of research and publications from a large number of researchers. Any of the individual papers along the way may be missteps; it is only in hindsight that a particular paper can be picked out as seminal or definitive.

That's OK.

It's also a stamp of relevance. Like the front page of HN vs new.

I'm not sure P<.01 helps overall, though I wouldn't cry if it happened.

1) small p-values require larger n's which will price some real studies out of happening, 2) it only makes dishonest research marginally harder and a little more obvious, 3) a stricter p-value may increase the misconceptions about how important they are, we already have enough B.S. flying around because "P<0.05, it's scientific now!".

I haven't totally taken a side but I find the idea of dropping p-values altogether to be interesting as well: https://whyevolutionistrue.wordpress.com/2015/03/05/psycholo...

> 2) IMO a much more stringent form of peer review is necessary.

That disregards the cost of review. Reviewing and repeating experiments uses resources that could be spent elsewhere. On the long tail of papers that are not high impact even if correct, that's probably not a good allocation of effort.

What I think would be better is that as a paper crawls up the impact curve (probably measured by citations), then additional resources should be spent validating and repeating it.

In theory, that's how the system already works since everyone wants to build on top of or refute famous papers. In practice, I don't know if it does.

I am not suggesting people need to necessarily spend more time doing the review, just accept less BS.

In many cases the bar is far to low.

Repeating experiments is not a wasted effort. "More than half of biomedical findings cannot be reproduced – we urgently need a way to ensure that discoveries are properly checked" - Elizabeth Iorns is co-founder and CEO of Science Exchange.

Science which can't be repeated, that is problematic.

[1] http://www.newscientist.com/article/mg21528826.000-is-medica...

Reply on news.ycombinator.com