FrontierCode 2.0

cognition.com

9 pointsRyan54532 comments

I'm super bummed about this. FrontierCode 1.1 is my favorite benchmark and this has dramatically different results, I don't trust it at all!

I sort of agree. Any benchmark that shows gpt-6 sol as better than gpt-5.6 sol is incredibly suspect.

Or I'm wrong and working with a model has an aspect to it that is not captured by any benchmark.