Set an experiment selecting 100 random software developers around the world and test this hypothesis. You're up to be surprised.
Nevertheless, most developers who wouldn't be able to "solve" a LeetCode challenge in a couple of hours even with access to Google, I bet would perform much better than o1 on real-world Github issues in their technical domain.
This highlights the essence behind OP's question about why focusing on Codeforces. And it shows me that "intelligence" involves a dimension that isn't logical and we don't understand yet.
> AlphaProof, a new reinforcement-learning based system for formal math reasoning
> AlphaGeometry 2, an improved version of our geometry-solving system
"There are 7 cats in a playground and 61 plushies, ..." (insert basically the same problem, requiring the same solution)
Well... Then the LLM shall be able to solve it.
And many people will consider it's a novel problem and hence a resounding success.
I mean: it is a success, but not anywhere near as impressive as most think.
Now, given the token encoding I think naive letter counting is not something we should expect from LLMs, but still serves as a nice reminder to actually ensure the test/validation data is not part of the training data.