Hacker News Reader

Comment by ethelward | Hacker News Reader

The more you work in the field, the more you discover how underwhelming the details in biology papers are. They give results without detailing the algorithms, hence destroying reproducibility, they hide datasets behind confidentiality, presents large-scale graphs without precise data, and so on.

Generally, reproducibility in biology papers is but a far away dream.

View on HN

>They give results without detailing the algorithms, hence destroying reproducibility, they hide datasets behind confidentiality, presents large-scale graphs without precise data, and so on.

Almost all science disciplines do this if the prestigious journals won't police it. Peer-reviewers who cared would quash these during referee periods if they wanted to, but they need to publish their data-less work as well.

Only a few journals across all of science care about that kind of thing. A few economics journals force algorithm and full datasets to be published.

It is still awful but far better now than a few decades ago. The absolute worst is Nature/Science from ~2000. For someone trying to figure out what is going on (rather than just believe what the authors claim) most of those papers are not even worth reading.

Any time before the advent of "supplementary materials" pretty much meant that you had nothing more than a highlight of the method. It was terrible indeed.

However, even today I find that roughly half the publications are not worth the paper they are printed on, or the bandwidth to download them.

Check out pre-1940 papers (the year may be later depending on exact sub field). I've seen that they used to make it a point to include all the info (including raw data) to the point it was practical. Somewhere along the line the attitudes went wrong, I blame NHST personally.

NHST ? What's that ?

I explained it and provided some references in an earlier post here: https://news.ycombinator.com/item?id=13483055

Here is another reference you could check: http://andrewgelman.com/2016/02/04/the-notorious-n-h-s-t-pre...

The problem with top-ranked journal more likely arises from page limitation. The amount of data needed for a paper published in such journals with so little pages and figures allowed basically means no one could have enough space to put enough material in the main content. And then who really takes a serious effort in supplementary materials?

With regards to method section of paper, while it is an important part for reproducibility, it is not writing in great details in top journals. The reasons I think lie in it is mostly in the supplementary material, the multiple methods used make it crazily long, furthermore applications of the commercial kits and highly-automated machines make it not necessary for researcher to write in great details.

Anyway, I agree that papers from Nature/Science are hard to read sometime, especially without reading the supplementary.

Reply on news.ycombinator.com