Take everything with a pinch of salt. Maintain healthy skepticism.
I think this needs to be clarified to a large section of reddit and hn members more than any place else on the internet.
I think this needs to be clarified to a large section of reddit and hn members more than any place else on the internet.
Skepticism is fine, and critical thinking should always be on with no exceptions, but HN is more prone to armchair quarterback bleeding edge science with no qualifications to do so (and predictably comical results), rather than uncritically accept results.
I myself am more likely to do the former than the latter.
You should remain skeptical, you should question results, and you should be capable of understanding the data and responses. You should understand uncertainty that's often unquantified yet still exists. You should apply critical thinking as well. You should also understand when you're way out of your domain and realm, potentially misunderstanding the situation or missing critical information in your analysis.
That's exactly what the article is about. It's fine for scientists to propose weird and outlandish theories with just enough analytical duct tape to ensure they aren't completely absurd. But those theories are meant to be analyzed more deeply and critically before anything is done with them, not just blindly assumed to be true because somebody published a paper on it and nobody refuted it yet.
There's a lot of statistical tricks that are tough to detect. Like is your sample truly random and representative? How could you even prove that? Like the rat part study purporting to show that drugs are only really dangerously addictive if you're also socially isolated and miserable. It's cool, and I kind of want it to be true, but it seems it's actually highly dependent on exactly what kind of rats you have and where you got them. Similarly, most of the time when you analyze a bunch of statistics, you're looking for a conclusion like that it's 95% likely that A is caused by B. That's only 1 in 20 though, so you should actually expect that if you do the same check for B through U, and none of them really cause A, you will falsely find that one of those letters causes A. But nobody published a paper for checking C through U, so you can't tell.