Hacker News Reader

> 1. Report the fact that disagreement exists. Phrased differently, stop taking academics at their word when they claim there's a consensus. Do some basic web searches to find people who disagree. Get quotes from them. Stress on the word people; not just other academics but literally anyone disagreeing on scientific grounds eliminates the claim of consensus. Bloggers are fine.

This is how you get journalists to report disagreements that don't exist in reality. For example, whether the earth is flat, whether climate change is real, etc.

> 2. READ the papers.

Literally nobody reads the papers. A huge amount of news isn't even investigated. A large number of news these days is regurgitated from other sources.

Journalists love reporting on flat Earthers even though their beliefs have no impact on anything in the real world:

https://news.google.com/search?q=%22flat%20earth%22&hl=en-US...

Realistically, journalists like to report on fringe or weird beliefs so they can laugh at the people holding them, and dislike reporting on serious disagreement with things they want to be true.

Agree that almost nobody is reading the papers, outside of random tweeters and bloggers. Journalists might as well start, though. Reading obscure documents is a part of the job, classically at least.

You can still look at credentials.

If there is disagreement among people with appropriate background, experience, and education, it can be reported as a legitimate disagreement. The journalist should be evaluating the credentials of sources, but not what they say, and not on the basis of whether the journalist personally agrees with them.

> This is how you get journalists to report disagreements that don't exist in reality. For example, whether the earth is flat, whether climate change is real, etc.

Correct. Part of the problem is that there is disagreement about whether a disagreement exists. Cranks believe that there is a vigorous debate about flat-earth/evolution/climate change, and scientists don't.

Do how does one (journalist) objectively determine whether an issue is settled or not?

Great comment.

One thing I'll add: if you don't have specific training in the field in question, just ignore any use of "scientific consensus" to justify an argument.

The "scientific consensus" trope is just dressed-up appeal to authority, and even if there is such a "consensus", it's almost never broad enough to be applicable to whatever pop-science journalism thing you're reading.

Also, even within the hallowed halls of academic science, most scientists are just repeating things they've heard other people say. Unless the "consensus" is amongst scientists who have spent their entire career studying the specific question (and by "specific", I mean...hyper specific, not just "in the same field", and certainly not something meaningless like "epidemiology"), this kind of thing just devolves into a popularity contest. You'd be shocked by how many PhDs just confidently repeat whatever silly thing they saw that morning in the New York Times.

Also, since I'm already seeing the meme appear...people are waaaaay too worried about "amplifying fringe voices" these days. News flash: if you don't know what you're talking about, then you can't possibly know what you should or should not be "amplifying". Stick to what you know, be modest about what you don't know (which is most things), and let the facts sort themselves out over time. Science only works if contrarians get a voice.

> Science only works if contrarians get a voice.

This is more of the rather tired "contrarians are always right" meme that seems to crop up constantly on HN.

Sometimes a contrarian is right and the accepted consensus is wrong. But that doesn't happen only because the contrarian position is contrarian, it's because the contrarians brought receipts. They applied proper scientific rigor and came up with a falsifiable theory that fits empirical observations and is sufficiently predictive. They also set out to disprove their hypothesis.

Not all contrarians need a "voice". It's not worth anyone's time to rebut yet another unfounded and stupid perpetual motion theorem or electric universe bullshit. It's far easier to spam stupid contrarian ideas than to produce real rigorous scientific output.

As Carl Sagan said, "They laughed at Columbus, they laughed at Fulton, they laughed at the Wright brothers. But they also laughed at Bozo the Clown."

> This is more of the rather tired "contrarians are always right" meme that seems to crop up constantly on HN.

No, it isn't. I literally did not say that, I didn't mean that, I don't believe that, and trying to spin it that way is a tortured way of reading the very sentence you quoted.

Contrarians don't have to be right to require a voice for the system to work.

> Not all contrarians need a "voice". It's not worth anyone's time to rebut yet another unfounded and stupid perpetual motion theorem or electric universe bullshit.

The point is, you aren't smart enough to know the difference. Nobody is. The way I know that science is working is because I can see the all the disagreements and judge for myself. Efficiency isn't the goal.

But since you're concerned, I spend exactly zero percent of time time worrying about perpetual motion or electric universes. Even if I did spend time on this, that's my choice, and who are you to tell me otherwise?

Folks who want to protect "my time" from "unfounded theories" are rarely as interested in in my time as they are about censoring things they don't like.

> Efficiency isn't the goal.

Well, it shouldn't be. But in case your research is informing "urgent" policy decisions, it's hard not to get frustrated and just want those who disagree to shut up.

In most things science I doubt you can "judge for yourself". In a field where you have some expertise, perhaps.

At least I don't have the knowledge (not just technical but also the lingo-related aspect) to read any paper or judge between any different positions on non-trivial things.

That's fair, but that means it's all the more dangerous to blindly accept things as facts. The field is important too; the more objectively data can be gathered and analyzed, the more trustworthy the research is. There are always issues, of course.

"Have an open mind, but not so open that your brain falls out."

> Science only works if contrarians get a voice.

I completely agree, but it's counterintuitive. There's a part of me that thinks "science == reproducible, observable fact". But it really is much more (er, less) than that. It'd be nice if there was a different word for the "not irrefutable" parts (i.e. almost all of it).

I'm not sure what scientific thing would be reported on in popular media, there was a scientific consensus that was reported on and was wrong. Anything from the past 3 or so decades you can point to?

I agree with your prescriptions, but worry the point might've slipped by.

> READ the papers

The point of my original comment is that most journalists don't have the educational background required to do this. And everything else flows downstream of that problem.

The issue is not that they can't do it, they just don't want to. Journalists with no science background are happy to write quite technical fact checks of articles - even articles written by scientists - when those articles are contradicting something the journalists are already invested in.

And often the problems don't need specialist knowledge to spot. The before/after images purporting to be of surgery in this article can be detected as fraud by anyone:

https://blog.plan99.net/fake-science-part-i-7e9764571422

No expertise needed. Blatant stuff like that is more obvious than we'd hope.

The issue is that journalists could do better if they wanted to, but they don't want to and there's no system of incentives in place that would make them want to. There's no such thing as a journalism license they could lose, and publishing nonsensical science articles doesn't hurt their careers in any way because nobody expects better of them. After all, they're just journalists with no STEM education...

Even if you somehow forced science journalists to all get STEM dual majors, it still wouldn't make them care. They'd still take the path of least resistance and pump out slop. The only way to make them care is to put them under editors that enforce standards. But how do you make a publication care enough to hire editors that care? Even state funded university press departments notoriously sloppy. If they can't uphold standards, I doubt any organization can.

I don't think you have any idea of the kind of time pressures journalists are under, and how little time they have to devote hours or days of deep research to each and every story they work on. At best you can get that for a large story if you're an investigative journalist, but very few have that privilege.

All of the journalists I know personally would love to have more time to dig into stories, to do more background research, to spend more time following them up, and to get to understand a subject more deeply. None are given that opportunity in today's high pressure, short-staffed, and cut-to-the-bones newsrooms.

it's rare for "layman" to find true errors in papers, and you weren't a layman when you did your investigation, as you say in the article.

That said, 90% of all papers contain at least one important error that brings the conclusions into question. Note that even great papers that established long-accepted truths contain important errors, see both https://en.wikipedia.org/wiki/Oil_drop_experiment#Controvers... and https://en.wikipedia.org/wiki/Oil_drop_experiment#Millikan's...

Similarly, the original 3D structure of DNA from W&C was actually "wrong" but I truly doubt any laymen (laypeople) could have determined that by reading the original paper (which is a paragon of clear and simple scientific reporting).

Although it's true I wasn't a layman, it turned out my own expertise in bot fighting had no utility to reviewing the paper. The review was done as a "layman", because the sociological subfield of twitter bot research has nothing to do with bots :( Anyone could have written that and actually most of the paper takedowns in that subfield are done by a team of a CS prof (with no specialism in bots) and a data journalist.

Understood. At best, though, I think these takedowns are based primarily on "technical invalidation": identification of one or more faults in logic, presumption, study design, or execution, which brings enough question to the results/conclusions that it's best just to assume the paper isn't worth reading.

(put another way: many papers that contributed what appears to be new, true knowledge contain factual errors, some of them invalidating, yet the paper turned out to be correct, in terms of its conclusions, in the long run)

wrt 1., there are an unfortunate number of cranks for every discipline. It would be funny to see journos publish the emails that every faculty member who releases their email in the university phonebook gets as Serious Disagreement* though.