Hacker News Reader

Comment by ltbarcly3 | Hacker News Reader

ChatGPT is completely incorrect, almost like it has no actual insight into the question... The 90 that did respond may or may not be representative of the rest of the students.

Example:

I send out a questionaire about plans post incarceration to 120 ex inmates. I get 90 responses that are very positive, and conclude that 100% of inmates have positive outcomes after prison. In reality 30 ex inmates didn't respond because they had committed suicide, or been murdered, or were back in prison.

View on HN

Extreme differences between those who answer a survey and those who don't is obviously always an area of concern. That said, under most circumstances, the alternative is you pretty much have to not bother with data (or do some qualitative research--e.g. interviews) or you have to go with what you have and maybe discuss potential areas of bias.

The topic for this questionnaire is "student life, academics, athletics", which is an important part of the question when determining if non-responses are important. In this case ChatGPT gets it correct, but explains the reason very poorly.

No it is not correct.

"the 90 students who did respond to the questionnaire are representative of the entire population"

How can anyone know this without knowing the size of the freshman class? For that matter, how does anyone know that 120 would have been a good number?

The more people you survey, the higher the confidence that the questionnaire is representative of the entire population, or the 'confidence interval'. Since the population was randomly sampled & non-responses don't correlate to the topic questions, you can assume that the 90 students are representative of the entire population with +- some confidence interval.

Yes, but we know neither the size of the population nor the confidence interval had the full 120 students responded.

Secondly, non-responses may actually correlate to the topic questions! For all anyone knows, students may not respond because

- too busy studying

- too busy partying

- too busy with intramural sports

- too disillusioned, feeling that it will not make a difference.

Etc

Reply on news.ycombinator.com