Why epidemiology refuses to reduce a ~80% false positive rate
lesswrong.com
lesswrong.com
For instance, the first studies on cigarette smoking and cancer were done in 1930's and they showed a relationship between the two. They were purely correlational. They could of been spurious. Even if we could do a randomized trial, it probably wouldn't be ethical. However, these initial studies paved the way for more research. Now we know with a high probability that the two are related. If we followed the author's advice where would we be?
There are also parallels to start ups. If we always followed the "hard evidence" how many new innovations and insights might we have missed?
> Even if we could do a randomized trial, it probably wouldn't be ethical.
What is unethical is not running a trial. We don't know anything and what we do know is frequently falsified, no matter how confident we think we are. This is the whole point of 'evidence-based medicine' - there are countless treatments where people went 'oh it'd be unethical not to use this awesome treatment' and made snide jokes about randomized testing of parachutes. But nicotine has many benefits (http://www.gwern.net/Nicotine#performance) and it's not clear to anyone who was not raised in the 20th or 21st century without the benefits of indoctrination that tobacco is bad on net. This is why we do trials. I am reminded of one quote on the topic:
> 'One day when I was a junior medical student, a very important Boston surgeon visited the school and delivered a great treatise on a large number of patients who had undergone successful operations for vascular reconstruction. At the end of the lecture, a young student at the back of the room timidly asked, “Do you have any controls?” Well, the great surgeon drew himself up to his full height, hit the desk, and said, “Do you mean did I not operate on half the patients?” The hall grew very quiet then. The voice at the back of the room very hesitantly replied, “Yes, that's what I had in mind.” Then the visitor's fist really came down as he thundered, “Of course not. That would have doomed half of them to their death.” God, it was quiet then, and one could scarcely hear the small voice ask, “Which half?”'
And besides, are you seriously trying to defend the last 70 years of bad epidemiology with one example which it is not clear even matters to the statistical question?
> If we followed the author's advice where would we be?
We wouldn't have pro-smoking people pointing out the absence of randomized trials and ambiguous animal results, a respected statistician like Fisher pointing out the curious correlations undermining the case for causation, or the smoking industry able to cast doubt and delay regulation for decades. (Ethical my ass!)