Hacker News Reader

Comment by raegis | Hacker News Reader

He's the example I have my students do in class. Roll a die repeatedly and tally the results. You'll get a (roughly) uniform distribution of 1s, 2s, 3s, 4s, 5s, and 6s.

Now to illustrate the CLT, you roll a die 50 times, and average the result. AND your 300 classmates do the same. If you tally the 301 averages, the distribution of the averages will not be uniform but bell-shaped, with average (approximately) 3.5.

The CLT says (roughly) the distribution of the averages will be approximately normal, regardless of the original distribution.

View on HN

Short and concise, as things should be. Thank you. :)

Here’s something that’s bothered me for a while. Why is it necessary to roll it 50 times? Why can’t I roll it just once? Does each X_i have to be a sample greater than 1 and if so is there some kind of minimum required for each random sample? Does CLT still hold if each random sample’s size is 1?

As the number of samples approach infinity the mean of all the iid samples will become normally distributed.

So it will neither hold strictly for n=50 or n=1, but n=50 may be a better approximation of infinity :)

As I understood it each X_i is a random sample and the n refers to X_n, that is, n random samples. But what is the size of each sample? In the parent it was 50. The dice is rolled 50 times by each person i, and the outcomes are given by the random variable X_i. Also, there are 300 people rolling a dice, so n=300, thus 300 random variables each composed of 50 dice roll outcomes. I’m curious why each X_i has to be a sample of size 50 and can we just have each person roll the dice 1 time? Maybe we need 50*300 people to roll the dice one time now? Does the CLT work when each X_i is a random sample of size 1?

Each person has to roll their die a number approaching infinity times, then the reported means will be normally distributed. Now you will almost always get something quite indistinguishable from normal with n=50 throws, but not with n=1.

If you plot the means of the 300 dicethrowers for n=1,2,3,4... dice. You will see the distribution of the means take on more and more of a bell shape. But fully normal it becomes only when n approaches infinity (and then it will be a quite spiked bell of course :))

So, to be clear: the CLT talks about behavior when n approaches infinity. The CLT can be used approximately before that, but it's a very bad approximation when n=1.

That's a good question, and it goes back to the definition of the CLT and the distinction between a distribution vs. a sample. We get to pick how we define the terms for our problem!

So in the original example, we know that if we choose X_n to mean an individual die roll, then the random variable X is not uniformly distributed. However, the CLT tells us that the average of many rolls is indeed normally distributed.

So if you pick n=50, then the average number of pips will resemble a normal distribution. What's the point of having 300 students do it then? It's so we can gather evidence that we're correct :)

Let each individual student do a 50-roll run. We know each run is i.i.d, so let Y represent the distribution of the average number of pips, and Y_1 ... Y_300 represent 300 samples - we can graph the empirical distribution of Y and test it for normality.

However, going back to your actual question, what if we had each person roll just 1 time? Is the outcome normal?

It depends on what you mean - how do we aggregate the 300 individual dice rolls? If you take each individual dice roll and tally it up (how many 1s, how many 2s, etc.), you will find that the distribution is still uniform - so no, a 'sample size' of 1 was not sufficient.

However, what is normally distributed is the average of those 300 dice rolls. However, in this case, we only get a single 'sample' (something close to 3.5), so we lack evidence that the CLT holds (even though it does).

Same goes with your other question - what if you had 50 * 300 people roll the dice? The average number of pips over 15k die rolls is approximately normal, but you've only drawn a single 'sample' in this case.

Going a bit further, this implies that what matters is not just how many dice we throw, but how we choose to define the meaning of those throws - for 15k throws, you can choose to think of it as 15k samples if a single throw, or 15 samples of (the average of) 1000 throws, or anything in between - just pick the definition that's useful.

Ok great explanation. Going to your last paragraph, what does this mean “in practice”. Say I have a dataset of 15k people throwing a dice 1 time. Can I group it into 300 random samples of 50 dice throws and now say I have 300 random samples and apply the CLT? Each random sample should be i.i.d no?

I was just curious if one can go the other way around, because usually you only have 1 sample of size 15k and not 300 samples of size 50. If you have the raw data it’s just 15k samples of size 1 or 1 sample of size 15k, depending on how you look at it.

Then I could be wrong here, but doesn’t the proof of the CTL also assume that each random sample is the same size? So you can’t have one sample of size 30 and another of size 20 and another of size 25, etc? Each of the 300 samples must be size 50?

Yes, each dice roll is i.i.d, so you can choose to look at it as 300 random samples of 50 throws each, or any other choice of N.

Put another way, what you have is 15k dice throws, and the fact that they were thrown by 15k, 5k, or 300 people can be ignored, if you choose to. In fact, it may be useful to 'shuffle' dice rolls into new 'samples' - that gets in to the use of resampling and bootstrap techniques.

And also yes, for CLT to apply, each sample must have the same N.

> I’m curious why each X_i has to be a sample of size 50 and can we just have each person roll the dice 1 time?

Smaller samples have higher variance and their means probably won’t converge as fast to a normal distribution.

“Samples” of one, in isolation, have no defined variance and you can’t use them to infer anything.

By the law of large numbers, the sample mean approaches the population mean as the sample size gets larger. The law of large numbers goes hand-in-hand with the central limit theorem.

Ok, but the real pdf is cut off at 1 and 6, while the normal distribution extends to +- infinity and can never model the cutoffs, no matter how large N is (though of course those tails get less significant).

like the other comment points out, the measured rv is shifted and scaled. re tails: there's a sqrt(n) that scales the measured distribution so that it in the limit it has tails at +/- infinity.

Reply on news.ycombinator.com