Hacker News Reader

Comment by BeetleB | Hacker News Reader

To be explicit, and using an example similar to the one in the article, if your CI is (2, 40), with the center being 21, there is no reason to believe that the true value is closer to 21 than to, say, 3.

To provide an extreme case, during the Iraq war, epidemiologists did a survey and came up with an estimated number of deaths. The point value was 100K, and that's what all the newspapers ran with. But the actual journal paper had a CI of (8K, 194K). There's no reason to believe the true value is closer to 100K than it is to 10K. Or to 190K.

View on HN

You're right, from the frequentist definition of a confidence interval (2,40) we can't say the the true value is more likely to be closer to 21 than to 3.

But we can't neither say that the true value is equally likely to be closer to 21 than to 3.

The point is that, from the frequentist definition of a confidence interval, there is nothing at all that we can say about how likely the true value is to be here or there.

It could be 3, 21, or 666 and there is nothing that can be said about the likelihood of each value (unless we go beyond the frequentist framework and introduce prior probabilities).

>The point is that, from the frequentist definition of a confidence interval, there is nothing at all that we can say about how likely the true value is to be here or there.

Yes - sorry if I wasn't clear. I did not mean to imply that each value in the interval is equally likely (and looking over my comments, I do not think I did imply that).

The complaint is that the article is stating otherwise as fact.

>One practical way to do so is to rename confidence intervals as ‘compatibility intervals’

>The point estimate is the most compatible, and values near it are more compatible than those near the limits.

They simply are not in a frequentist model (which is the model most social scientists uses). I agree with the main thrust of the article in that there are many problems with P values. But I am surprised that a journal like Nature is allowing clearly problematic statements like these.

I don't know enough about the Bayesian world to be able to state if his statement is wrong there as well, but if it is correct there, it is problematic that the authors did not state clearly that they are referring to the Bayesian model and not the frequentist one.

(Not to get into a B vs F war here, but I remember a nice joke amongst statisticians. There are 2 types of statisticians: Those who practice Bayesian statistics, and those who practice both).

> I did not mean to imply that each value in the interval is equally likely (and looking over my comments, I do not think I did imply that).

When you said that "the values at the boundary of your interval are as credible as in the center" you kind of implied that, which is why I asked.

I won't defend the article being discussed, but you opposed their statement that "the values in the center are more compatible than the values at the boundary" with an equally ill-defined "the values at the boundary are as credible as in the center".

I do not read my statement to imply uniform distribution.

What I meant was "there is no reason to prefer values at the center more than values at the boundary" based on the CI (there may be external reasons, though). To me, this is equivalent to your:

>there is nothing that can be said about the likelihood of each value

Ok, we agree. "As credible as" implies uniform "credibility". "More compatible than" implies non-uniform "compatibility". Without any clear definition of "credibility" or "compatibility" it's impossible to interpret precisely what are those claims supossed to mean.

> there is no reason to believe that the true value is closer to 21 than to, say, 3.

I find this very silly, since if we ditch the arbitrary 0.95 and go with 0.999.. confidence interval of [-998, 1040] for example. How can one say that one cannot tell if which value is more likely, 21 or 1040?

If this is an actual limitation of the frequentist model like you said, everybody should be a bayesian thinker then. And the "confidence interval" is just a quick way to communicate how wide and where the posterior bell curve is.

Reply on news.ycombinator.com