This comment essentially says "the prompt is faulty because the AI didn't answer it correctly". This bakes the correctness of the AI into the tautology that it always answers "good" prompts, which are defined as those it answers correctly.
If you're prompting ChatGPT (which is already loaded with prompt-engineering cheez-whiz mind you) to reason as a computer, why use words like "imagine"? why phrase things as a question? I think the fragility the author is observing here is extraneous to the actual underlying technology, and a factor of the UI that's been stacked-on and purpose-built for softer-purpose use, not hard academic research such as this.
Case study, even GPT2 can reason about the stack https://transformer.huggingface.co/share/bQXbmeyBZD
"Can reason" is a generous interpretation of "got the right answer in one instance".
So it's no surprise we didn't teach AIs yet!
Edit: I don't understand why experts say it failed when they "asked it to reason". It doesn't have pictures of TVs on cats to produce variations of, right? How does "reason" enter into it?