Hacker News Reader

This is a really good point, it was the first thing I noticed as well. 17 responses isn't even enough to throw a margin of error at, the entire survey is effectively a margin of error of 100%.

If it really was only 17 responses that's the equivalent of asking five people on the street if they like hot dogs and then deciding based on their answers to open up a laundromat in Brooklyn.

View on HN

Once you add in a proper confidence interval, 17 responses on a yes/no question is enough to make decisions like this - as long as you know in advance what your target numbers are. The downside of the small sample is that your margin is going to be extremely large, of course.

Using Clopper-Pearson at95% confidence tells us that between 23.3% and 70.8% would answer yes to this question. I do agree that is probably still too broad to decide to kill a business, but if you know you wanted a 60%+ (for example) positive response from this audience to hen that sample is enough to cause some serious worry.

It's then breaking them down into subcategories. Look at the confidence interval on the posted charts: +/- 80% on the high income responses this survey was interested in. I.E. worthless.

Reply on news.ycombinator.com