I tried to play with this problem (on text-davinci-003, which I prefer to ChatGPT).

It performs considerably worse than it does on many harder problems I gave it before. It seems to capture some architectural weakness in GPT.