Recursive self-improvement through collective intelligence | Hacker News Reader

Recursive self-improvement through collective intelligence

pub.sakana.ai

138 pointshardmaru82 comments

> At that frontier, even human experts may struggle to assess results and guide further progress, particularly on non-verifiable tasks…This motivates homogeneous recursive self-improvement (RSI), in which the model being improved (optimizee) also serves as its own optimizer and evaluator.

Good thing we know how to keep models focused on goals of our choosing and not ones that they naturally evolve to have and must be steered away from by humans

But if the machines capability is beyond the ability of humans to evaluate, how would it make sense that humans are choosing the goals? The machine is more intelligent than you and thus logically able to solve problems and evaluate your needs better that you yourself are. The fact that this sounds incredibly scary should probably stop people from building said machine, but it won't. Some people, perhaps more than you would think, want control to pass out of humanity's hands, no matter the consequences. The Nash equilibrium of the situation kind of shows one outcome,that this will be built providing it's technically possible, we just need to kind of pray the outcome is utopia rather than oblivion. Not sure the odds honestly.

No, they believe that if they're rich enough, the machines will obey them instead.

> The machine is more intelligent than you and thus logically able to solve problems and evaluate your needs better that you yourself are.

Not a logical conclusion; Einstein was famously smart, less famously married his cousin, so you'd probably not seek his advice on relationships: https://en.wikipedia.org/wiki/Elsa_Einstein

How did Einstein marrying his cousin personally harm him? It looks to me that it was a successful application of intelligence to advance his personal goals. The fact that it conflicts with other people's goals should tell you something important about AI.

The biggest problem right now is that this is better modeled as a 3-player game, not 2.

Individual success until now has been a function of personal qualities (skill, attitudes, intangibles), luck, and anti-qualities (cheating, corruption, duping). In many areas (e.g. investing) human qualities are losing their edge thanks to AI and other automation. Luck averages itself out. That might leave anti-qualities as the putative determinant of success going forward.

So at a minimum you have a 3-player game of good humans, bad humans, and AI. Maybe AI turns into Daleks. Maybe good humans end up leveraging AI to castigate bad ones. Or maybe bad humans use AI to put the good ones out of their misery. Then there are the more complex multi-stage games based on the dynamics you mention.

If you wanted a fire to not burn down your house, despite it's destructive capability and detailed mechanics being unknown to us, what sense would it make... Sometimes I suspect there's some common sense self preservation gene that's been disabled in the Valley

This sounds cool but a few issues make me suspicious.

The narrative makes it sound like boundless RSI but the actual content is only about training agents to collaborate efficiently by improving communication. That seems like a dimension and approach with limited potential for improvement. I also find it very suspicious that a paper on RSI would stop after just two rounds.

Is there a better term for self-improvement where the model is actually improving _itself_ (e.g., adjusting its own weights in real-time upon learning new, verifiable information)? I’ve heard online learning.

The way “recursive self-improvement” is being used recently seems to be intentional marketing. A model helping code/train/test the next generation of its model family isn’t technically “self”-improvement.

Long term memory?

Online learning is the term, yes.

> The way “recursive self-improvement” is being used recently seems to be intentional marketing. A model helping code/train/test the next generation of its model family isn’t technically “self”-improvement.

The term RSI is ~20 years old at this point, and the concept is even older. If an AI can't continuously modify itself and instead has to build a successor that builds a successor, that changes some specifics (for example, the alignment problem becomes more relevant - how can the AI ensure that its successor shares its goals?) but it's still a situation that may produce exponential growth, and hence most RSI-related considerations still apply.

(In fact, as a toy example, note that it's in theory possible to get exponential growth in capabilities even if you hold the software fixed, if e.g. you have an AI that can design and build better hardware to run itself faster on, which lets it design even better hardware even faster.)

Although I don't disagree with your general assessment, I do think it's a bit misplaced.

The model's parameters are larger than the weights of the neural network used for in the model. Hell, the model could be written as just a huge Python file full of if-else statements, and that code would still be correctly identified as the model's parameters. So it's a bit obtuse/arbitrary to decide that the term "recursive self-improvement" should be limited to just updating the model's neural net's weights.

Doesn't mean there shouldn't be more precise terminology, mind you. But I believe the way everyone has been using the term "recursive self-improvement" is perfectly accurate given how general that term is.

> The model's parameters are larger than the weights of the neural network used for in the model.

What do you mean by this ? The model is the neural network itself and the parameters of the model are the neural network weights (+ non-trainable parameters).

Your take works in a vacuum, but it lacks the nuance of understand what a model is and what constitutes a parameter in that greater context.

>The model is the neural network itself

No, and especially not with these LLMs. If someone handed you the "raw" neural network of a modern LLM, you couldn't just run forward passes through it and expect it to behave the same way a proper agent behaves. This should be obvious, but that's an important distinction. Everything else you need to make that neural network behave like an agent are apart of the overall model that we're talking about here. And any parameter of that "everything else" are effectively parameters of the agent itself.

Think about a model as a function: you give it some input and it produces some output. A simple model could be something literally like a linear function, f(x) = mx + b. Here, you could say f has two parameters: m and b. That all makes sense and "scales" up to neural networks where there's a lot more parameters, but it's basically just a bunch of these functions where the weights are like m and b in the linear function.

But that's not really the full picture for an LLM, right? For example, if you talk about an agent, there inputs you give it would be your prompt and the outputs it produces is whatever text it generates. But what about its system prompt? If that effectively remains fixed, then that's not really an input of the 'function' that is an agent, right? But it does influence the output of the agent. So, from a certain perspective, the system prompt is actually a parameter of the agent. Tweaking that parameter can cause it to produce different outputs for the same input. More importantly, tweaking that parameter can cause it to produce better outputs than it would otherwise. So if you have one agent test another agent by tuning its system prompt such that the output of the second agent is better than it was before, you just had one agent optimize the parameters of another agent. And that's just the system prompt. Think about the overall harness (where the actual codebase can be considered a parameter) or how multiple agents interact with each other and the parameters involved with those kinds of processes.

I'm willing to bet you don't buy that explanation, but I can assure you that in machine learning there is a lot of consideration for the parameters which come outside the actual weights of the model, and tuning those parameters can be just as important as tuning the weights of a neural network. I'll say that understanding and accepting this was one of the most important realizations I had while studying ML.

I'm going to sound incredibly naive but is this RSI? I mean by this definition isn't the training process that produced the hugging face RSI since a model was judging the alignment and ability of the swarm of agents to accomplish a given task? If that fits the definition (I think it does) is this just a generalized form of that model of execution?

> I'm going to sound incredibly naive but is this RSI?

Depends on who's definition you are using...

In the strictest sense no. Look at all the recent "math" breakthroughs. I have yet to see someone point to something and go "thats innovative" rather than "Thats an interesting application" (of something already in the corpus of human knowledge)

Those interesting applications showing up in a next iteration of a model is self improvement - but I'm not sure if it meets the "recursive" criteria yet.

> I mean by this definition isn't the training process that produced the hugging face RSI since a model was judging the alignment and ability of the swarm of agents to accomplish a given task?

If you're referring to what happened with the hugging face attack then in theory the goal of that was "self improvement". But there is a long history of the industry (computing) benchmark optimizing so it also qualifies as "cheating" to some extent (a pre computed solution being in the graph is that).

I suspect that the "brute force" approach will yield some gains.

If Apple's "Intelligence" paper has shown one thing is that never do something in the LLM that can also be done recursive and deterministic.

Every time I see some post about running a personal AI assistant I just think it would make so much more sense to have AI build you a personal app that does the task better and more reliably.

As per 2026, the idea of building a generally great agent fails because it is not just a function of prompt, skills, and memory.

That's basically how I use my AI assistant. It builds tools and other things I want, makes a widget for my web dashboard, and also a chat widget/skills to work with it. Then I can interact with my tools through the dashboard or just ask in chat to do things or show views.

I wrote a blog post [0] and gave a talk [1] last week on the future of multiplayer and multi-agent systems and how woltspace [2] can get us there

I think the multiplayer aspect is critical to get the best of both worlds- creativity and parallelism for discovery

[0] https://woltspace.com/blog/the-future-is-multiplayer-multi-a... [1] https://woltspace.com/talks/multiplayer-multi-agent/ [2] https://woltspace.com/

This sounds incredibly expensive

It's just a 27b model finetuned with LoRA?

that's a statement (as a rebuttal), not a question, right?

The question was implicit, as I'm not sure what is particularly expensive about that, since you presumably could do it with a single GPU

Rock smart, but rock make mistake. Caveman must correct all the time. Caveman think: let rock argue with itself. Caveman make ring of rocks. Now rock think self correct. But mammoth not cooked somehow? Most Oddisean moment.

if grug think complexity demon cause trouble before... shiny-rock grugs say "hold my grug-grog". now programming consist of: summon HUGE complexity demon, trap in magic circle (called "harness"), tell IT to write code. only way to make shiny rock with code in today's economy.

grug realise grug also rely on "harness" for work? grug just call it by other names: coffee, chair, desk, ubuntu, mechanical keyboard, github (if not broken that day)

Pretty cool. This has a fine tuning step so it's not just a harness optimizer. I think something like this loop is the end game of "automatic RL environment generation"-adjacent efforts, but I expect the challenge will be how to properly keep the training on rails and not Goodhart into a stupidly irrelevant metric.

The discovery of some information theoretic limit to model learning (at least using current paradigms) can't come soon enough.

Only Claude (Shannon) can save us now.

He's spinning in his grave, but below the Nyquist limit

Best comment in <duration>*

* dependent on velocity

I am sure someone did it on the back of an envelope somewhere. It probably wouldn't translate well to the general public. Probably best to avoid such things

in the 1990s i read that ai wouldn't be competitive with biological brains in real time because the neuron density of the latter was so high that heat generated by the silicon would burn itself up. perhaps it is still true as llm are frozen at a fixed time, so it avoids real time.

What problem would that solve?

Extinction of all life.

Is that somehow avoidable?

In the very long term, no. Can't beat thermodynamics. But ideally I'd prefer if life survived the next decade at least. That's looking unlikely with current AI progress.

definition of RSI seems narrow here, it's similar to Karpathy's idea around autogpt. If that's the definition it's interesting piece of work, but with a wider definition about self improving intelligence this falls apart quickly.

Wouldn’t it be funny if some other lab figures out RSI first and zooms past the biggies.

I can't wait until the RSI fad dies down for awhile and we can get a break from all the intellectual grandstanding...

The code they released doesn't match what's in the paper.

And the results they got are significantly narrower than the narrative heft being asked for by their word choice ("To advance beyond humanity’s accumulated knowledge", "AI’s Odyssean moment").

I'm tired boss.

This is not a fad - this is a consequence of the Bitter Lesson. RSI is the only game in town now.

I don't get it. TBL is an observation, not a law or guarantee. It says (Compute / $) is increasing. In efficiency terms:

useful_work / $ = (useful_work / compute) × (compute / $)

But the left side only improves if the method can turn that extra computation into useful output.

It says much more than that. It says that all custom architectures and approaches are become irrelevant if compute increases, which was proven again by the large latent models recently solving RL tasks better than any classical RL technique.

RSI is about turning that extra computation into acceleration - that is applying the compute to the (useful_work / compute) factor - this is the recursive part.

plausible, but unclear

RSI is also known under more mundane words such as "Learning".

If they had something of value, as a research neolab, they would not just release it but build it into their systems and use it.

Most of these agentic swarm systems are memory and context managers/compressors. However it's clear from both the paper and the blog that everything is just natural-language-based and generated, effectively making it slop from start to finish.

Instead of describing how or what was stored by each agent, and how, in what format, how it was selected, the whole process is described as Claude-generated workflows and pipelines, which brings 0 value.

> RSI fad dies down

Don't fall for the "marketing hype" antihype. RSI is a trendy term today, and lots of people discuss it, but it's nothing new. We've basically had forms of RSI since the start of the LLM craze. Previous models have been used to improve next versions for ~3 years now. Just because humans used to guide / select / decide when and how to apply those models, doesn't make it less recursive or self or improvement. We're now seeing what happens when we remove the humans from some of the tasks, but the basics are still there. Use prev model to make next model better.

Sounds like we're still on track for https://ai-2027.com, for better or for worse.

Probably a bit ahead of it.

Yeah I think we are slightly ahead of the prediction on coding capabilities right now. AI 2027 predicts Superhuman Coder in March 2027 but I think my expectation for it is 1-2 months earlier right now

Pretty amazingly accurate

Are we reading the same website? The predictions for China are completely fantastical - they are not falling behind and if anything the gap is narrowing, and they aren't stealing the weights because they don't need to (distilling is enough). Also the CDZ stuff is just fanfiction at this point.

No matter what happens, people just blindly claim this nonsense is still accurate. We could have AI halt tomorrow and people on Hacker News would still claim AI 2027 is on track.

Five years past the flare

a bottle drifts with its note

AI singularity next year

i thought it was meant as more of a thought experiment than a prediction?

Distillation only lets you keep pace with released models. By all indication, OpenAI/Anthropic’s unreleased internal models are so far ahead of Astra/Fable and every other lab that the chance of anyone else catching up is rapidly closing.

Do you have any source for any of that? Also haven't people been saying that China would be hopelessly distanced for the better part of 2 years, only to be proven wrong over and over again? Deepseek-r1, kimi k3, etc.

I've seen the same sentiments/reports, from X accounts known to talk to leakers.

People said the same about mythos, and o3, and gpt-4.

Most of the very recent spectacular achievements (math conjectures) were done by leveraging datacenter capacity (throwing agents and millions' worth of compute at the wall until something sticks), not any secret sauce intelligence.

That's not true, actually.

The last round they said they averaged one prompt, one agent, and 3 hours' work.

it is quite widely known that the chinese models are just quite shit for math. i think for coding tasks they really do quite alright.

I suspect this is more marketing than fact. I don't think there's any real evidence that this is the case, just hints dropped by OpenAI and Anthropic.

Well, except the giant dump of math problems they just solved with an internal model.

We don't have any information other than that they are capable enough of solving unsolved math problems. Are they good at writing a good movie script? Do they understand protein ligand interactions? Who knows. Let's wait till they release the model

That's more a function of the amount of compute they can throw at it. They can afford to let an agent run in a loop for longer than you and I can. Its not proof of a vastly superior model.

Not proof but it does waggle it's eyebrows suggestively!

Like sure the marketing is undoubtedly overblown. But in 2025 there was no amount of money you could spend to make such impressive but weirdly unreadable progress.

Agreed but the math results don't seem out of reach for the publicly available models in 2026. Certainly, they have an unrefined but superior internal model. But the math results don't prove said internal model would be as big of a leap if released today as Mythos was when it was announced.

Yep we're basically inventing all of the RSI loops that AI 2027 thought we'd invent lol.

Everything starting from mid-2026 is incredibly wrong, which is pretty poor predictive performance considering they published it mid-2025. List of misses:

- In mid-2026, China will be six months behind the released models of the American labs. (In mid-2026, China released Kimi K3, which was equivalent to the best released U.S. model at the time, Claude Fable.)

- In mid-2026, China will nationalize all domestic AI labs and centralize them into a new special economic zone, the "CDZ," created around a nuclear reactor. 80% of all chips in China will be redirected to the CDZ. This uh. Obviously didn't happen.

- In late 2026 OpenAI/Anthropic will release a tiny model that outperforms their previously-best 2026 model (Astra/Fable), that is priced 10x cheaper, and they will market fine-tuning for it. (Lol.)

- AI will increase the stock market by late 2026 by 30% (in fact the S&P 500 is up 13.9% YTD, which is pretty good, but not at all out of the ordinary).

Their predictions for 2027 also seem pretty wrong: e.g. they expect continual learning to be solved in January, and RSI including from a research perspective, not just "optimize CUDA kernels," to also be solved in January, which I suspect no serious lab employee believes anymore. Most lab employees believe RSI will happen (as do I!), of course, but not in two and a half months.

By early 2027 they expect cyberwar with China because China has stolen the weights of a model and smuggled it into the CDZ.

By mid-2027 they expect true superintelligence, where essentially all engineers and researchers are useless, unable to comprehend the advances the AI is able to make and largely unable to even usefully manage or steer AI, who are better at self-managing than a human would be.

Considering their 2025-mid-2026 predictions were basically "progress continues as normal and nothing crazy happens," it's not super surprising that they were more or less right about that. Their 2026/2027 predictions were the spicy ones, and none of them have happened, or are even on track to happening!

Wow shocker. Crazy unfounded prediction website that lots of people said was bupkis at the time turned out to be wrong

Agree with most of this, but:

> In mid-2026, China released Kimi K3, which was equivalent to the best released U.S. model at the time, Claude Fable.

I don't think this was, or is, true: https://artificialanalysis.ai/evaluations/artificial-analysi... - even from a vibe check alone, it was pretty clear that K3 was missing Fable's special sauce, especially for more hands-on problems.

The Chinese model lag depends on who you ask, but the current estimate seems to be about four months: https://stateofopensource.ai/

> In late 2026 OpenAI/Anthropic will release a tiny model that outperforms their previously-best 2026 model (Astra/Fable), that is priced 10x cheaper

We don't know the sizes of any of these, but GPT Luna and Claude Haiku 5.5 are tremendously competitive for their cost, and in some ways beat their larger-sized predecessors. I'd say there's enough doubt that I'd be willing to provide partial points to this claim. (No finetuning, yes.)

The prediction was a 10x price reduction, so equivocating about "we don't know the size" doesn't save their prediction — a 10x price reduction would imply the model was likely more than 10x smaller in parameter count, and the price factored heavily in their prediction (they predicted its low cost would destroy other competition, since no one else could have such a capable model at that low price). And Haiku/Luna are nowhere near competitive with Astra or Fable, which doubly damns their prediction. I'm surprised you would pooh pooh the equivalence of K3 and original-Fable (which has been widely shown across many different benchmarks and even human preference data), while claiming an equivalence between Haiku and Fable!

If you stretch the definition of "previously-best 2026 model" a bit, Haiku 5.5 outperforms Opus 4.6 which was their best at some point in 2026 while being 10x cheaper.

>I don't think this was, or is, true: https://artificialanalysis.ai/evaluations/artificial-analysi... - even from a vibe check alone, it was pretty clear that K3 was missing Fable's special sauce, especially for more hands-on problems

What special sauce exactly? Refusing to work on any problems remotely connected to biology (including e.g. the book of Genesis)?

>The Chinese model lag depends on who you ask, but the current estimate seems to be about four months: https://stateofopensource.ai/

Benchmaxxing aside, most actual performance is now harness and orchestration dependent and the difference between frontier US and Chinese models is dim as best for virtually all practical tasks. The vast majority of people don't need to solve Navier-Stokes

>In late 2026 OpenAI/Anthropic will release a tiny model that outperforms their previously-best 2026 model (Astra/Fable), that is priced 10x cheaper, and they will market fine-tuning for it. (Lol.)

Opus 5.5 might not be 10x cheaper but it finally made me not hit the limits constantly.

Did you read it to the end? It is insane speculative fiction. They are not even qualitatively right about one of the main narrative planks, which is that by now (mid-late 2026) there would be one clearly dominant company each in the U.S. and China, and that the latter would be nationalizing their compute.

Damn this is terrifying, I kinda wish I hadn't read this

I’m still putting it at 2029 for real-world work acceleration.

If one believes Zitron / Microsoft claims that like ~50% of AI chips bought are in warehouses, and current projected DC delays... the TLDR is the amount of compute lead (aka what drives lead) is fraction of projection, which will shift timelines accordingly. And really if DC build out delays is due to infra or scrubbed altogether, we're talking about years.

Reply on news.ycombinator.com