Hacker News Reader

Comment by apo | Hacker News Reader

Sensible advice overall, but I completely disagree with these:

> Before you begin reading, take note of the authors and their institutional affiliations.

and

> Beware of questionable journals.

Institutional affiliation and journal imprimatur should have no bearing in science. These are shortcuts for the lazy, and they introduce bias into evaluation of the paper's contents.

Even more than that, dispensing advice along these lines perpetuates the myth that scientific fact is dispensed from on high. If that's the case, just let the experts do the thinking for you and don't bother your pretty little head trying to read scientific papers.

If the author's approach to reading a paper only works by checking for stamps of approval, maybe the approach should be reconsidered.

View on HN

They aren't shortcuts for the lazy, they're shortcuts for non-scientists who aren't capable of fully evaluating the science alone. If you're capable of objectively peer reviewing a paper, you're not the audience of this article.

> They aren't shortcuts for the lazy, they're shortcuts for non-scientists who aren't capable of fully evaluating the science alone.

It's a shortcut fraught with potential for deception, as even a casual glance through a site like Retraction Watch will demonstrate:

http://retractionwatch.com/

I'm not sure what you mean by "evaluating the science." A scientific paper should present a hypothesis, the author's best attempt to disprove the hypothesis, and an interpretation of the evidence gathered in the processes of testing the hypothesis. There's going to be a back-story, and it's likely to be quite involved.

The article does a good job of presenting a method for navigating a paper on this basis. I don't see what checking credentials adds to the process. On the contrary, it may do harm.

While we may find the high profile cases featured on Retraction Watch mainly in high impact journals, that's precisely because unscrupulous people deem these journals worth it to cheat to get into. Nobody cheats to get their paper into the International Journal of Architecture, Engineering and Nursing Science - because it and it's ilk are utter pieces of crap that will accept anything, up to and including randomly-generated text and pro-Sri-Lanka-highly-racist-UFO-conspiracies (real example). Teaching laypeople to avoid these is a very good idea.

On the contrary I totally agree with the author on the point regarding journal—while noisy, reputable journals generally have better editing and stronger peer review. Of course it isn't the case that less credentialed journals / authors or bad, it just increases the probability of wasting your time reading something charitably only to find massive theoretical or methodological shortcomings.

Bias = priors. Use your priors.

>Institutional affiliation and journal imprimatur should have no bearing in science.

They shouldn't, but they do. This is reality. Industries producing research are often industries that would benefit from the research being biased in their favor. This is an important element to consider.

It's not about dismissing certain affiliations, it's about being conscious of institutional bias.

I don't find institutions valuable in this regard as well, unless you're looking for a potential conflict of interest there.

Journals is something else, mostly because finding out if it is a "real" journal can help to sort out a lot of crap. If it's a journal where you just pay to get stuff published without any peer-review that tries to look like a real scientific journal, it's pretty safe to just stop reading.

Using the prestige or impact factor of the journal as a guide for quality is likely misguided as well. Though it is a warning sign to me if a paper makes claims that look like they should be publishable in a prestigious journal, but it actually appeared in some journal nobody knows.

I agree with you. Your comment totally reminded me of SCIgen [0].

[0] http://news.mit.edu/2015/how-three-mit-students-fooled-scien...

You can beware of something and still read it. In general, I think it's pretty sound advice. It's less about turning your nose up at someone because they're not a professor at Harvard, and more about asking whether a group with ulterior motives has spun up an institute and is essentially self-publishing "research".

I check these all the time. It's not my job to verify who was first to publish and who deserves credit for novelty. There are groups that do that which divy up acknowledgment. It's my job to get up to speed with as accurate information as I can with the limited time I have.

Institutional affiliation: https://en.wikipedia.org/wiki/Aspartame_controversy

"Searle had submitted 168 studies on aspartame"

Reply on news.ycombinator.com