Comment by lutusp | Hacker News Reader

Comment by lutusp | Hacker News Reader

Parent Full thread

> The poster he was responding to made an idiotic statement:

> "In statistics, σ refers to an area under the normal distribution defined in terms of standard deviations (1σ = 1 standard deviation)."

Not only is that not idiotic, that's the default definition in a statistical context (see below). Anyone can argue that σ is just another Greek letter with no special significance, but that require one to ignore the context in which the term is used.

Link: http://en.wikipedia.org/wiki/Standard_deviation

Quote: "In statistics and probability theory, the standard deviation (SD) (represented by the Greek letter sigma, σ) shows how much variation or dispersion from the average exists."

> He was just stating that the standard deviation is a general concept of spread, that any distribution has, and not just the normal one.

Again, this disregards context. When nontechnical people ask what the significance of 5σ is to scientific statistical analysis in physics (which is how this thread got started), there is precisely one answer.

* "In statistics, σ refers to an area under the normal distribution defined in terms of standard deviations* that's the default definition in a statistical context

No, you're quite wrong about that. greycat is correct in his/her corrections of what you're saying. In statistics sigma is used to represent one of two things:

- a parameter of a probability distribution, typically one which influences the spread of the distribution

- a measure of dispersion in an actual data set, which may be an estimator of a parameter in a probability distribution

Neither of those things necessarily involve the Gaussian density.

In physics it seems that "sigma values" are used as a scale to measure p-values, so that instead of saying 0.0000003, they can just say 5σ. But the critical point here, which your comments seem to be missing, is that there is no distributional assumption being made; there is no implication that the Normal distribution describes any data-generating process, merely that the probability of an equal or more extreme value of a test statistic under some model, is the same as the probability of observing a value more than 5 standard deviations from the mean under a Gaussian model.

>> * "In statistics, σ refers to an area under the normal distribution defined in terms of standard deviations* that's the default definition in a statistical context

> No, you're quite wrong about that.

It is the default, actually. There are plenty of exceptions to the default, but it certainly is common in the context of experimental physics, the present context.

http://physicsbuzz.physicscentral.com/2012/07/does-5-sigma-d...

Quote: "But what does a 5-sigma result mean, and why do particle physicists use this as a benchmark for discoveries?

To answer these questions, we'll have to look at one of the statistician's oldest friends and C-student's worst enemies: the normal distribution or bell curve."

> ... your comments seem to be missing, is that there is no distributional assumption being made ...

Do read some experimental physics -- see what assumptions are made. Here is how a physicist maps a sigma value to a p-value:

http://i.imgur.com/pcjr6cN.gif

If this wasn't a discussion of the analysis of the outcome of a physics experiment, I would be more likely to accept these digressions.

> Do read some experimental physics -- see what assumptions are made.

lutusp, the way in which you are using the word "assumption" carries a very high risk that people will misunderstand you. The critical point here is that the physicists are nowhere using the Normal distribution as a modeling assumption. They are not suggesting that the Normal distribution is a reasonable model for any real data generating process in their problem domain. They are simply using it as a scale, like Celsius of Fahrenheit. There's a crucial philosophical distinction there that, even if you get, your readers will not.

You disproved your own point.

In statistics, σ refers to an area under the normal distribution defined in terms of standard deviations (1σ = 1 standard deviation).

In statistics and probability theory, the standard deviation (SD) (represented by the Greek letter sigma, σ) shows how much variation or dispersion from the average exists.

I think you're missing the difference.

If the unit normal distribution is the distribution under discussion, there's no difference. In that case they become the same. Here's how a physicist computes a p-value given a sigma value:

http://i.imgur.com/pcjr6cN.gif

Note that the unit normal distribution is the underlying context.

Reply on news.ycombinator.com