Hacker News Reader

Comment by thejacenxpress | Hacker News Reader

small unrepresentative sample size. a problem many studies have issues with.

Actually, compared to most studies I'm used to, this is a commendably high sample size. 2a and 3a have N=~250. 2b is 518, and 3b is over 2000. When was the last time you saw a psychological study with sample sizes this big?

Aside from just size though the major problem is the gender disparity of ~80% women in most of the studies. I didn't see mention of ethnicity but I imagine it's overwhelmingly white.

Yeah. This is a big problem, because your average psychology student is a very particular kind of person - young, middle class woman that is bad at maths and simply wants to get a degree from something.

Before I get called out as -ist, let me explain. Psychology is one of the few "default" subjects - i.e. something you pick when you don't know what do you want to major in. It's also a stereotypical feminine subject, hence huge gender gap. The other popular "default" choice is economics, which tend to draw those proficient at maths.

The point being, psychology students are a very particular subset of the population, and thus it's difficult to generalize results from them to everyone.

I agree with the unrepresentative part, but absolute sample size numbers don't tell you anything. For some experiments, N=100 is more than enough, and for others N=10000 isn't even close. Did you look at the p-values or any other confidence measurements?

Statistically, doesn't a sample size of 2000 get you pretty close to +/- a 5% margin of error?

I'm not sure what you mean by 'statistically' above.

In general you can't say whether a sample size of 2000 will get you within a specific margin of error without additional information. It very much depends e.g. on what the hypothesis you're testing is or what you're trying to estimate.

And all are students, who are going to be generally middle class and young.

This is very common across most psychology studies from what I understand.

> In the Journal of Personality and Social Psychology, the premier journal in social psychology – the subdiscipline of psychology that should (arguably) be the most attentive to questions about the subjects’ backgrounds – 67% of the American samples (and 80% of the samples from other countries) were composed solely of undergraduates in psychology courses (Arnett 2008). In other words, a randomly selected American undergraduate is more than 4,000 times more likely to be a research participant than is a randomly selected person from outside of the West.

Source: http://hci.ucsd.edu/102b/readings/WeirdestPeople.pdf

Reply on news.ycombinator.com