Hacker News Reader

Comment by bluecalm | Hacker News Reader

16 adults is quite good. If you polled them well (that is randomly) and you obtained some result in all of them then it's significant evidence that that the result is true for very significant part of the population.

Sure, you can't show that it's true for everybody but having a result like: "only 0.1% chance that it occurred by random if 2/3 of the population reacts this way and way less if it's half of the population" is very nice result to have.

If something occurs for 16 randomly chosen people (or even 15/16 or 14/16) you are virtually certain it occurs for very significant part of the population which is often the goal of the study - to justify further research.

View on HN

If you polled them well (that is randomly)

It's not random. They were "sixteen overweight/obese men and women 30-66 years old, with a BMI between 27–50 kg/m2".

How important is it that this study had no control (and thus no randomisation or blinding) and had a small sample size, and that some of the authors have written books about this diet?

What would a control group have added? If you boil water in a beaker, do you need a control beaker where you monitor the temperature but don't light the Bunsen burner?

Also, how would you run a double-blind test of the foods people are eating? It's pretty hard to disguise steaks ;-)

Most nutritional studies are observational, so the numbers are not very reliable. In this case, "subjects were provided with all food for 18 wk, which was prepared and packaged by staff in our research kitchen." That's one reason why the numbers were so small, though it would have been better if they'd been bigger.

The most reliable evidence would come from controlling what people ate and what they did. I'd have thought this would be something that could be done with volunteer prisoners.

Controls are always essential in this type of research. Placebo effects abound. Maybe the people knew they were on an increased fat diet and so exercised more?

I'm all in favor of controls, in general. But how exactly would a control help in this specific case? How would it work?

Nutrition studies will often use people as their own controls because of the significant variability between people. Nobody is willing to fund nutrition research with "perfect" studies, so researchers do what they can (especially when they are known to have hypotheses counter to the orthodox view).

On their books, I think that would be a ridiculous thing to reject people's findings for. Almost every academic has written a book at some time. Instead, we need to recognize the possibility of a bias inherent in every researcher and look to the study design, execution, and conclusions to see if they hold. Phinney and Volek are in a different class when it comes to their books and their ideas than is, say, Robert Lustig.

Reply on news.ycombinator.com