Hacker News Reader

Comment by eanzenberg | Hacker News Reader

If you're doing a measurement, why have a null hypothesis? Alice should sample the population at random, take the height measurements, calculate the average, plot the distribution, calculate the variance. If the distribution is not sufficiently smooth then continue to take measurements until it's smooth, or unchanging. Then Alice is done discovering all there is to know about the height distribution of the population she sampled. Same with Bob.

View on HN

> Alice should sample the population at random, take the height measurements, calculate the average, plot the distribution, calculate the variance

A student-t test is basically a lossy encoding of that information you describe (the sample mean, the variance, and the sample size).

> If the distribution is not sufficiently smooth then continue to take measurements until it's smooth, or unchanging.

From a frequentist standpoint, this would be considered sloppy and bad methodology. You don't keep sampling until you get the results you want (or until the null hypothesis is invalidated).

That said, as I mention in a comment above, the fact that you can keep sampling until the null hypothesis is invalidated (and that it is mathematically always guaranteed to happen eventually) is a big problem with the concept of hypothesis testing in the first place.

Creating an 100(1-a)% interval estimate of the population mean based on the sample's mean and variance is equivalent to examining the set of all values which produce a p-value below a%. Your view of the way Alice and Bob should be making inference about the true mean is actually exactly what hypothesis testing is. It's just that there is often a "status quo" belief about the parameter which has important theoretical implications. And academic papers are usually about that belief specifically, so it makes sense that they talk about it as a "hypothesis test". And that's the logical headline, the statement about the specific value. But standard deviations and distributions are often given, making it trivial to create the interval as a reader.

Reply on news.ycombinator.com