It's presented as a limitation when GPT provides an incorrect answer.
Both of these behaviors are literally the same. We are sorting them into the subjective categories of "right" and "wrong" after the fact.
GPT is fundamentally incapable of modeling that difference. A "right answer" is every bit as valid as a "wrong answer". The two are equivalent in what GPT is modeling.
Lies are a valid feature of language. They are shaped the same as truths.
The only way to resolve this problem is brute force: provide every unique construction of a question, and the corresponding correct answer to that construction.