Hacker News Reader

Comment by jbay808 | Hacker News Reader

> This is putting the cart before the horse, for sure. The reason why you only know the mean and the variance of your data is because you chose to summarize your data that way.

No, it's not... A Gaussian is the best way to represent your knowledge of a value if you only know the mean and variance of its value.

So if you start with a stack of data and compress it down to a mean and variance, you've discarded most of your knowledge, and are left with a Gaussian as your best guess representation.

Yes if you were to boil it down to different summary data, like a max and min, you'd end up with a different state of knowledge and a different distribution.

But given a mean and variance, the Gaussian is your best choice of distribution, and not because of the central limit theorem, but because it has maximum entropy on those constraints. You don't always even have access to the source data in the first place, maybe just the summary statistics.

View on HN

I would like to push back against this in favor of the original comment. The context of this remark within the article is:

> I was extremely confused as to why the Normal (Gaussian) Distribution pops up everywhere—in kurtotically-ignorant financial market analysis, in nature, everywhere. Thinking about it, the prevalence of the Gaussian is actually rather abnormal. Can you guess why it’s everywhere?

This is not a "compression of data" question. It's not an "uninformed distributional choice" question.

It's a "why is this distribution prevalent in Nature" question.

In this context, I think the CLT gives a better answer. There are a lot of averaging processes in Nature, and due to the CLT, averaging of independent perturbations must give rise to normal distributions.

It's possible to perhaps go a step deeper than the above. In some physical systems, you can look at the second moment as an energy -- like the voltage-squared in electrical systems.

In this case, due to a-priori finiteness of system energy, the gaussian distribution can make a claim to being "inevitable" by the maxent argument in OP. ("In a system characterized by finite energy E, what is the least informative distributional constraint?")

Because additive processes are common. If the variables were multiplied together instead of summed, you'd get a different asymptotic distribution.

You'd get the lognormal, since multiplication is just the exponentiation of summation.

> But given a mean and variance, ...

The subject of my comment is, "Why are we given mean and variance?" If you take, "We are given mean and variance" as a presupposition, then you are having a different conversation.

The big problem with the maximum entropy argument is that if you apply some transformation to your data, you will end up with a different maximum entropy distribution. For example, you may choose to express your data in terms of rate (events / time) or period (time / events). Maximum entropy won't help you here, you have to have some kind of theoretical understanding of the underlying process that justifies your choice.

The same is true for normal distributions and mean / variance, but it's such an "obvious" choice that people forget to justify their models. My experience is that the premise of the CLT is much easier to justify, and you can use that to support your use of the normal distribution.

> if you only know the mean and variance of its value.

And why are they the only things you know? Most often, because they are the things you asked for previously... rather than, as OP hinted, e.g. some facts about quantiles.

Reply on news.ycombinator.com