Do you have a source for this? That's interesting (if true).
https://techcrunch.com/2025/01/19/ai-benchmarking-organizati...
Is anything here actually false or do you not like the conclusions that people may draw from it?
From a primary source:
"OpenAI did not use FrontierMath data to guide the development of o1 or o3, at all.... we only downloaded FrontierMath for our evals long after the training data was frozen, and only looked at o3 FrontierMath results after the final announcement checkpoint was already picked."
> If an MMLU question asks about an abstract algebra proof, is it cheating to have trained on papers about abstract algebra?
This kind of disingenuous bullshit is exactly why people call you cheaters.
> Generally, I don’t think anyone here is cheating and I think we’re relatively diligent with our evals.
You guys should follow Apple's cult guidelines: never stand out. Think different
"OpenAI did not use FrontierMath data to guide the development of o1 or o3, at all.... we only downloaded FrontierMath for our evals long after the training data was frozen, and only looked at o3 FrontierMath results after the final announcement checkpoint was already picked."
From a primary source:
"OpenAI did not use FrontierMath data to guide the development of o1 or o3, at all.... we only downloaded FrontierMath for our evals long after the training data was frozen, and only looked at o3 FrontierMath results after the final announcement checkpoint was already picked."