If you did’t bother to write it, I shouldn’t be bothered to read it.
Perhaps AI agents can have their own publications and magazines where they are the chairs and associate editors and reviewers.
You don't think this changes when the thing in question is a proof of a STEM problem no human has ever been able to solve?
We'll see what the final slop rate is, but the three papers they retracted yesterday were for a trivial sign error. If they didn't catch that, that means OpenAI isn't bothered to put in the minimum effort of sifting through their own garbage and making sense of it.
It's not like they needed to hurry out this release before carefully vetting. They're just "hacking" the math system and disrupting the work of thousands of researchers to create a gigantic RL dataset for themselves.
Really, what's the bloody hurry?
They could have released 1-5 papers, worked with researchers to understand what methods work and what don't, how to prompt the models better, how to build better guardrails for reasoning, etc. And give those researchers access to latest models and empower then to solve thousands of problems!
Instead OpenAI wants to piss all over the city to claim territory and now human mathematicians have to go around cleaning up that slop, only so that OpenAI can made some bullshit statement like: math is solved [mistakes are next].
--
Imagine someone gave you a million line PR claiming to have vibecoded the operating system of the future (or whatever your application domain). Would you drop all your other work to focus on this? And they generate enough PR that your manager and company leadership and public all start pressing you to accept it quickly? Guess what, it's your lucky day! You have not one, but 700 breakthrough PRs!
If it verifiably works and solves important outstanding issues, then quite possibly yes.
Not to mention the math paper (on HN yesterday) which pointed out that OpenAI might have verified the wrong thing, in their Navier Stokes proof.
It's really not as cut and dried as you (and software/AI folks more generally) think it is.
PS: If a human mathematician had to retract three of their papers a day after posting publicly, they'd lose all credibility and their mathematical career would be all but finished. That social incentive structure is the field's immune system against slop. You're basically asking them to turn off their immune system, and for unclear gains (other than OpenAI's grandstanding).
If a human published 700 papers and only 3 (or 30) ended up having significant errors, I'd call that a pretty good batting average, considering that the typical rate of errors may be around a third (https://lamport.azurewebsites.net/pubs/statistics.pdf). But for some reason people hold AI output to an absurd standard where if it's not 100% perfect then it's useless.
You're basically asking them to turn off their immune system
I'm not asking "them" to do anything. They can do whatever they want, including rejecting obviously useful tools. But then they shouldn't be surprised when they're quickly surpassed by others who don't share their ideological blinders.
The math peer review process can take months/years to decide on the correctness of a solution -- especially those written as sloppily as these AI proofs. Let's see what the numbers are once the dust settles -- how many of the 700 are correct, and how many of those are valuable.
----
We're seeing the math analogue of this: https://www.nbcphiladelphia.com/news/local/anthropic-ai-mode...
I don't think you have a consistent argument or stance.
It seems your argument is that nobody should be reading the "AI slop" proofs, because if they were seriously studied, a lot of them will turn out to contain errors, and you know this because people have checked and found 1% of them to be wrong, which you knew even before anyone checked them, because nobody should be reading those proofs in the first place.
At least you'd be consistent if you keep arguing that it's going to be a waste of time to check them. You knew it right? Why even bother to see what the numbers turned out to be?
If AI can solve such grand, outstanding math problems, and mathematicians argue these pure math problems are important, what’s the problem with them needing to read the output if they want to understand it?
The alternative you are proposing implicitly is even crazier. OpenAI should not release a proof that is most likely correct so that it doesn’t burden others. What? It’s not about that guy dude. It’s about the society TM. One can’t delay progress because a guy may be burdened.
“Guys plz don’t release this thing that is absolutely correct but I’m kinda busy with other things ok?”
The alternative is that they do the work to properly present the results. They spend billions of dollars in AI training and inference but can't afford to even cite the literature properly? They're doing the bare minimum because they're inly interested in doing a PR stunt.
Bare minimum is still _solving_ the open problem standing there for years. Nobody owns math. Nobody owns giving enjoyable proofs to someone else.
If you don't like to engage with OAI proof dumbs in current state, don't. Maybe others will. Or maybe _these_ mathematicians are afraid that _other_ mathematicians will do it. Just elitism and gate keeping.
Ok, I don't see the point of discussing with you. It's clear that you decided what to believe in and no evidence will convince you that reality is more complex. The proof is that you ignored all the nuances expressed here by simply sticking to your simplistic interpretation, without any explanation of why such nuances are invalid.
Hey! It is really nice that you have dropped solution for so many open problems. We appreciate it and would like to see more. However, can you please work on the output writing/explanation quality? It is hard to read and understand. We would be happy to independently validate these but current paper quality leaves much to be desired.
That's it. This would make paper quality an important target for next generation models.
If problem was unreadable proofs, that would have been the reaction. But that's not what we are seeing :)
FWIW, in many online platforms, many actual mathematicians who referee papers, have said the quality of the papers are not outstanding, but they are still better than average.
> FWIW, in many online platforms, many actual mathematicians who referee papers, have said the quality of the papers are not outstanding, but they are still better than average.
I don't know what "online mathematicians" you're talking about, but I'm a professional one, do reviews and discuss regularly with other mathematicians (some of them of high profile); no one thinks that these AI papers are above average in terms of writing. This is what even in the most well-known online platforms has been repeated over and over. It seems like you're biased on this one.
Proofs can be unreadable for more than one reason. Are these ones unreadable because the math is super advanced or because current agents suck at clear writing? Maybe a bit of both?
Do you disagree with this? For example, if openai had provided really readable proofs with utmost care but still dropped 400 at once, would there have been less outrage?
Absolutely. If they cared about properly citing the bibliography, sharing how their models work, etc, it would be so much easier to make an assessment of the situation, of what will be math in the future, and all those deeper questions. What we got instead? from those 400 papers, there are already 4 that have been found to be a copy of recent publicly-available papers. They spend millions for "the good of science", but they can't afford basic scientific ethics?
Less criticism, yes
There’s always going to be outraged people, but outrage isn’t the word I would choose to describe the positions of the mathematicians I’ve read on this topic, including TFA. There’s a lot of optimism mixed with frustration that something important is missing