Comment by pkoird | Hacker News Reader

Comment by pkoird | Hacker News Reader

Science is only as good as it's honest. I get a result that's not as flashy but did I report everything correctly? Yes? Then it's good science. I get a result that's groundbreaking but I left out certain (potentially problematic) caveats in the report. Bad Science. Maybe a valuable result won't be recognized till much later, or maybe a relatively suboptimal result will continue to sub-optimally drive some key processes. Imo, it doesn't matter as long as we know them to be from good science.

The only problem that I can see from where I stand is that the machinary is consistently being disincentivized to produce good science. PhDs want to finish their research at any cost possible, Professionals are constantly under the publish or perish dilemma, and there is an increasing difficulty in getting enough reviewers to go through a manuscript with desired rigor. Not sure what'll fix it though. Perhaps efforts to promote good science as opposed to a great one like accepting publications for failed attempts (michaelson morley style), replication results of earlier works, or the general acceptance of the fact that research is difficult and one can not be expected to pull the figurative rabbit out of one's hat every couple of months?

> Not sure what'll fix it though. Perhaps efforts to promote good science as opposed to a great one like accepting publications for failed attempts (michaelson morley style), replication results of earlier works

Too often we try to solve social problems by "adding" something, whether it be adding an incentive or adding a program. I think to really solve the problem of publish or perish mentality, we first need to understand the root cause or causes of this mentality, then work to remove them. What I'm seeing here is humans being shepherded by enormous economic and social pressure to engage in selfish behavior for survival and/or social acceptance. Adding an incentive or a program therefore ultimately does not work because it does nothing about the fact that the humans are still largely enslaved by the aforementioned pressure. So, we must remove the pressure. Remove the pressure, remove the selfish behavior. But how to remove the pressure?

> But how to remove the pressure?

Since this is simple to answer (remove all requirements to publish frequently, and hope that a lot of journals die naturally after that), the real question is: how do we distribute funding to scientists without forcing them to frequently show their work?

I could imagine a world where every scientist (starting from Ph.D student onwards) is evaluated only e.g. on the basis of a biyearly dissertation-style report, which includes all (positive and negative) results, all data/metadata/code/analysis. Rapid communication of interesting results can still happen at conferences and the remaining journals.

But then who reads, reviews and ranks all this work? Who gets positions and funding?

The Carnegie Institution of Washington used to use this model — each year publishing a 'year book.' From 1902 through the 1980s, Institution funded scientists contributed detail reports — including figures and even new results — to the organizations Year Book. Year Books often exceeded 700+ pages.

Today, the Year Book is little more than a glossy fundraising document.

You can view the reports over the years: https://carnegiescience.edu/about/history/publications/carne...

Reply on news.ycombinator.com