Was this with GPT 3.5 or 4? If it was with 3.5, the analysis is entirely irrelevant in my opinion.
Many people just use GPT 3.5 because it's free, not realizing how much it sucks in comparison to newer models.
Many people just use GPT 3.5 because it's free, not realizing how much it sucks in comparison to newer models.
People use 3.5 for testing the capabilities of LLMs, and then conclude that LLMs are inherently bad at that task, when in reality there are better models.