Hacker News Reader

Comment by phamilton | Hacker News Reader

ChatGPT says:

  The best plan of action for the university is to use the 90 questionnaires that were submitted as the final sample. This is because the sample of 120 students was randomly selected, which means that the 90 students who did respond to the questionnaire are representative of the entire population of students in this year's freshman class. By using the responses from these 90 students, the university will be able to obtain accurate and reliable results from the questionnaire.

View on HN

Why should we care what ChatGPT says?

ChatGPT is completely incorrect, almost like it has no actual insight into the question... The 90 that did respond may or may not be representative of the rest of the students.

Example:

I send out a questionaire about plans post incarceration to 120 ex inmates. I get 90 responses that are very positive, and conclude that 100% of inmates have positive outcomes after prison. In reality 30 ex inmates didn't respond because they had committed suicide, or been murdered, or were back in prison.

Extreme differences between those who answer a survey and those who don't is obviously always an area of concern. That said, under most circumstances, the alternative is you pretty much have to not bother with data (or do some qualitative research--e.g. interviews) or you have to go with what you have and maybe discuss potential areas of bias.

The topic for this questionnaire is "student life, academics, athletics", which is an important part of the question when determining if non-responses are important. In this case ChatGPT gets it correct, but explains the reason very poorly.

No it is not correct.

"the 90 students who did respond to the questionnaire are representative of the entire population"

How can anyone know this without knowing the size of the freshman class? For that matter, how does anyone know that 120 would have been a good number?

The more people you survey, the higher the confidence that the questionnaire is representative of the entire population, or the 'confidence interval'. Since the population was randomly sampled & non-responses don't correlate to the topic questions, you can assume that the 90 students are representative of the entire population with +- some confidence interval.

Yes, but we know neither the size of the population nor the confidence interval had the full 120 students responded.

Secondly, non-responses may actually correlate to the topic questions! For all anyone knows, students may not respond because

- too busy studying

- too busy partying

- too busy with intramural sports

- too disillusioned, feeling that it will not make a difference.

Etc

So, option A :)

Option D

Reply on news.ycombinator.com