I think anthropic one is edited by humans, OpenAI drop just looks like how Astra writes (i know it's probably a different model producing this but still)
Makes me sad that the homotopical foundation people never got their shit together and now formalization is so mature but it's still done in set theoretic foundations.
I mean, the models (starting with Bel clearly and Astra probably--though Astra probably is worse than top humans in some areas) are superhuman at math.
I don't understand. The AI-math guidance counsel has recently explicitly argued against what you say in paragraph two, and which is why for the Kakeya problem there was no group formed. The different communities have given different suggestions. I'm not sure how much I agree with the first paragraph either given these facts.
The answer, as Noam Brown explained in the Dwarkesh interview, is that it is not emergent at all. They carefully selected reward functions in RL to incentivize cooperation in the last year in order to develop multi-agent systems which are faster and more reliable. However, I guess after it is baked into the model to cooperate, they can sometimes override instructions not to.
If you appreciate beauty and don't care about competing then these releases are purely good. Because you are not competing, so you aren't hurt by speed. And you are appreciating so you can appreciate more stuff.
I think now it's a good time for people to work on the Riemann hypothesis. There are actually a lot of ideas, no one knows if any of them goes anywhere--but I don't think it's completely impossible to make progress especially with new tools haha.
I think I'm mostly on your side (also I believed early it would get here), but what does it mean to "solve the field"? I'm pretty much as bullish as you get on AI but I'm sure I can come up with questions AI cannot solve. I think the space of problems in math is so large and the distribution of proof lengths so heavy-tailed, there's probably no way without dyson-sphereing the sun to solve all easy to state problems.
surely the LLM can do that. It is RL'd against some reward, if the known strategies are clearly suboptimal with easy improvement, it'll find it most likely
It's honestly just good at math. Even at theory building i wouldn't put it below 90 percentile. Just on some things it's superhuman already and some not yet.
Absolutely not the right way to deal with a rogue super intelligence. At a minimum it could implement some dead man switch when it's out and knows about impending kill switch
I believe this is false. They hack bc hacking has nontrivial initial probability (within range of behavior seen in pretraining) and that probability is being heavily rewarded in RL post training