Highlight negative results to improve science
nature.com
nature.com
So you end up with three numbers. The probability you find something given that there's nothing, often 5%. The probability you find nothing given that there's something (something being a 2 unit effect), often 20%. The size of effect you're using in these calculations (2 units).
The smaller effect size you're looking for, the more data you need. The lower error rates you're looking to get, the more data you need. So you make sacrifices and accept higher error rates. You make optimistic effect size estimates (making your true error rates even higher).
Using p < 5% as a positive result publishing threshold is bad enough. Using power > 80% as a negative result publishing threshold is even crazier. That's a 20% error rate for publishing negatives. Higher if you used optimistic effect estimates.
What this all means is yes we should publish negatives, but a negative worth listening to is going to be expensive. In a bayesian sense, it's the difference between me spending little money to learn that the effect of coffee on mood is probably between -1 and 1 (a bad negative result) vs spending a lot of money to learn that it's probably between -0.001 and 0.001 (a more useful negative result).
I actually trust negative results way more than positive results now- it's a proxy that you can trust the investigator. I wish we had a Journal of Negative Results.
You see this even in highly objective fields such as physics, where much progress comes from the previous generation simply dying.
Our own ego's, as well as the factors mentioned in the article, create problems much more than you might expect when you're young and naive.
Nobody's making progress because people are dying (we still see many vestiges of the Cophenhagen interpretation!)
Think it depends where you're coming from. Physics is objective if by objective you mean we can predict how the wavefunction will evolve according to the Schrodinger equation.
(Depending on who you ask) it might be less objective if you ask: ''But what's REALLY going on?'' Then you start getting into the interpretations of qm and different scientists will have different views.
I think it's at least likely that we'll be able to answer that question in a few decades (or at least move the ball forward). The view of Everettians, for example, is falsifiable and there are experiments going on as we speak trying to achieve that.
https://slate.com/technology/2013/05/weird-psychology-social...
What we really need is to decouple the publishing decision from the outcome. A way to do this is to use registered reports.