If this model is not benchmaxxed, I fear it's over for anthropic, openai etc. They are not unique and there will probably be many others with similar models very soon. It'll become a commodity and the price will be driven to the ground.
I understand that it's still useful marketing, say for IPO or hiring, but it's not clear to me that it really matters in the long-term that they fund this out of their pocket.
If OpenAI did it purely for marketing, I still think it was well worth it. It marked an important milestone in the history of computing (and also human history). It is on the same level as when IBM's Deep Blue defeated Garry Kasparov in chess, or when DeepMind's AlphaGo defeated Lee Sedol in Go. The search for winning moves in those games is not dissimilar to the search for paths towards mathematical proofs, and AI has finally proven that it is capable of carrying out deep, complex, and intricate research that exceeds human capabilities.
Besides, a Super Bowl ad costs about $8M, so it is not absurd if they spent a lot of money on this and became the first company to definitively demonstrate AI can surpass humans in mathematics.
This is already so good that 90% of my work has shifted away from codex to this. So, muse-wonder-full coming near opus 5.5 is kinda expected.
This is the entire point here, give it a couple years every nation on the planet with 20-40B in funding would have their own champions with post Fable level performance.
Was this sarcasm?
How many nations on earth have 20-40B for LLM training? At 40.2 B, Spain has the 15th largest military budget in the world. I doubt any nation will spend 40B on gradient descent.
Like peloton in cycling.
E.g in f1 you can get very close but difficult to pass unless you have a significant advantage.
Page 16, Section 1.4, Notes on coverage of unreleased models.
"a noticeable improvement on Mythos 5 for many tasks relevant to internal use but does not display a capability jump"