I interpreted it as a linear chain of events, end result being 28.
- we had 19 chairs in the storage room
- we used 10 chairs (removing them from storage)
- and then we purchased 9 more chairs (adding them to storage)
The answer from chatGPT is consistent with English and makes use of all the facts; either the question is testing for deception (intentionally sharing misleading information) or chatGPT made the correct inference, by assuming good faith and using all the provided information.
This question says something about us — that we’d assume deception ahead of unclear reference to a storage room by a good faith speaker.
The room is the room (what type of room is not relevant). There are 19 chairs in it. The 19 chairs most likely got there when 9 more chairs were purchased after 10 were used, but that is not relevant to the question: how many chairs are in the room? 19.
A := 19
B := 10
C := 9
What does A equal?
Possible answers:
- 19 chairs, least likely answer given the original
- 18 chairs, 10 of original 19 removed and 9 added
- 28 chairs, 19 chairs in the room with 10 in active use and 9 new chairs added
Just seems like a question a human would fail too. Probably opens a can of worms on interview questions and personal bias etc too in a more general sense. Not bias in the political sense, just how interview questions encode a lot of the asker's assumptions.
Edit: the only unambiguous phrasing I can think of is "19 chairs are in a room, 9 of the chairs are green, how many chairs are in the room?".
This is exactly why I prefer the precision of code over natural language for inputs (and why Google's botsplaining of "what you really meant to search" instead of just searching exactly the query as-is is so frustrating).
In a test, this seems like the kind of question they use to justify not giving a perfect grade because whatever answer you pick, they can claim it was the other one because both interpretations are reasonable.
Question: There are 5 dogs in a room right now. The previous day an additional dog was purchased and added to the room. How many dogs are in the room right now?
Answer: There are 6 dogs in the room right now. The statement says that the previous day an additional dog was purchased and added to the room, so the total number of dogs in the room right now is 5 + 1 = 6.
Edit: Ok a comment below pointed out that the question is still unclear. See below for the fix and the response. It is still incorrect:
Q: There are 5 dogs in a room on 1 January 2023. The previous day an additional dog was purchased and added to the room. How many dogs are in the room on 1 January 2023?
A: There are 6 dogs in the room on 1 January 2023. The previous day, one additional dog was purchased and added to the room, which means there are 5 dogs + 1 additional dog = 6 dogs in the room on 1 January 2023.
In other words, I would posit the probability of you having meant for the answer to be 5 low, because the question itself becomes trivial. For a system that needs to deal with many people asking questions, this robustness is helpful in my view.
In other words, it's not totally clear that time has not passed between sentences in your prompt. So the second instance of "now" could be a different "now".
The "robustness" you're perceiving in this case is just a mere coincidence, and doesn't reflect an aptitude for answering poorly phrased questions, but it does reflect a fundamental problem with these systems.