i tried out gpt 5.4 xhigh and it did meaningfully worse with the same prompt as opus 4.6. like, obvious mistakes
definitely not bologna, at least anecdotally :)
That is I get more variance between opus 4.6 and itself than I do between the sota models.
I don’t have the budget for statistical relevance but I’m convinced people claiming broad differences are just vibing, or there are times when agent features make a big difference.
either way, all that one can really rely on is the benchmarks, and those are easily cheated/overfitted to.
i think it's all very hard to quantify, so take my previous comment with a massive rock of salt