Comment by slr555 | Hacker News Reader

Comment by slr555 | Hacker News Reader

Meta-analysis: the non-study, study that says, well, anything you want it to say.

I am so tired of second rate journals running rehashes of other researcher's work and pawning it off as a "ground breaking examination of the data". Anyone in the field of medicine will tell you that meta-analysis is the least credible form of "research".

I find the Guardian author's first hypothesis most credible. There are a lot of bad therapists out there. It is well established in surgical studies that results for a procedure will be less positive in study sites that are not centers of excellence as the surgeons are less practiced and less skilled. Surgeons and therapists are not stamped from a machine. As the commercial says, "your mileage may vary". Put another way, what do they call the person who graduates last in their class in med school? Doctor.

For any medical ailment of any consequence you might face you will be well served to find the best and most experienced provider available to you.

On what basis do you say that meta-analysis is not credible? From my stats background, (but without any direct experience in meta-analysis) I would say that they could be problematic because the N is very small and so have a lot less to work with, and more temptation to use fancy statistical techniques with hidden assumptions. On the other hand, what do you do when you have 50 studies of varying quality, no one of which is has such a compelling methodology as to eclipse the others? In that case it seems like meta-analysis is the least bad option.

Your point is a good one but in medicine the issue tends to be that a lot of studies with varying controls and endpoints are being lumped together and presented as a coherent whole. In reality the distortions (and in some cases biases of the researcher) that are introduced leave the data often untenable. You are absolutely right that powering medical studies correctly is a critical issue. It is also critical that when making pronouncements about important treatment modalities that the research be of the highest quality.

Meta-analyses are often used in so creating so called evidence based medicine standards which are sometimes cost cutting efforts dressed up like research.

As a trauma surgeon recently said to me recently, "They said they designed it using evidence based medicine. They didn't say whose evidence".

Reply on news.ycombinator.com