Hacker News Reader

Comment by DanBC | Hacker News Reader

Science doesn't work like that.

To get meaningful results you need a large sample size, with a control group, with patients not knowing whether they're getting the new medicine or the old medicine[1] and with doctors not knowing either. Then you need to write that up and analyse it.

Many patients are happy to take part in such trials, but it's hard to get ethics panel approval.

[1] you tend to test against current best treatment, not placebo, because almost anything works better than nothing and by the time you test the treatment in humans you need to know if this is better than what you're already doing.

Ethics panels have some weird effects -

(http://www.badscience.net/2011/03/when-ethics-committees-kil...)

View on HN

That's how it works in the best case. Unfortunately, medical research is not a perfect science. Realistically, with many medical treatments, you can't have a double-blinded study. With chemotherapy, the side effects will often quickly tell you what treatment you're receiving, and a treatment like for example extracorporeal photopheresis can't ethically be placebo-controlled or double-blinded at all. What are you going to do, hook half your patients up to a machine that will do nothing but run their blood around a bunch of tubes for hours for no reason?

And you can't get a "large sample size" of patients with rare cancers or diseases by definition. Sometimes, the best you can do is a case study of ten patients who agreed to try your treatment because the other option was death. Quite a few medical advances have been made that way, actually.

Science is rigorous, repeatable observation. Double-blind studies and null hypotheses are excellent ways to achieve this. But case studies and low n studies are also informative - and are science. That science can only be control groups and null hypotheses is a myth.

Try and tell the medical fraternity that case studies should be eliminated because they hold no meaningful result and you'll be laughed out of the room.

Case studies are lousy because they are not rigorous, and often have not had any meaningful result.

See also knee arthroscopy for patients with osteo arthritis - many of these surgeries were performed before a trial with a control group was possible. That trial found that people who got the sham surgery did as well as the people who got the real surgery. Case studies are lousy, often harmful, medicine.

The thing is that you read case studies with this caveat in mind, recognising that they're statistically flawed. When you don't have better information available, case studies and low n studies give some direction, some previously observed information.

Reply on news.ycombinator.com