Recursive Language Models (RLMs) | Hacker News Reader

Recursive Language Models (RLMs)

alexzhang13.github.io

135 pointstalhof835 comments

Briefly, an RLM wraps an existing language model (LM) together with an environment that can dynamically manipulate the prompt that will be fed into the LM.

The authors use as an environment a Python REPL that itself can call other instances of the LM. The prompt is programmatically manipulated as a Python variable on the REPL.

The motivation is for the LM to use Python commands, including commands that call other LM instances, to figure out how best to modify the context at inference time.

The results from early testing look impressive at a first glance: An RLM wrapping GPT-5-mini outperforms GPT-5 by a wide margin on long-context tasks, at significant lower cost.

I've added this to my reading list.

A comparison to dSPY would be nice. cmd+f in the provided link doesn't bring any results tho...

An RLM is like a language model using DSPy plus all of Python to manipulate its prompt.

Sounds like unforgivable overhead for very questionable benefits, this whole LLM space is an overengineered slop, and everyone is jumping in building layers on top of layers of slop.

This is old news! Agent-loops are not a model architechture

I’m confused over your definition of model architecture.

Loops aren’t recursion?

Loops and recursion are fundamentally equivalent.

See e.g. https://textbooks.cs.ksu.edu/cc210/16-recursion/08-recursion...

Only if you have indexable memory that you can use as a stack, which in the context of LMs isn’t a given.

As another example, a finite-state-machine language can have loops, but it can’t recurse unless there is external memory it has access to in a way that it can serve as a stack. Regular expressions also fall into that pattern; they can loop, but they can’t recurse. For that you need a pushdown automaton: https://en.wikipedia.org/wiki/Pushdown_automaton.

Everything old is new again when you are in academia

This feels primarily like an issue with machine learning, at least among mathematical subdisciplines. As new people continue to be drawn into the field, they rarely bother to read what has come even a few years prior (nevermind a few decades prior).

This reminded me of ViperGPT[1] from a couple of years ago, which is similar but specific to vision language models. Both of them have a root llm which given a query produces a python program to decompose the query into separate steps, with the generated python program calling a sub model. One difference is this model has a mutable environment in the notebook, but I'm not sure how much of a meaningful difference that is.

[1] https://viper.cs.columbia.edu/static/viper_paper.pdf

If you would setup an RLM, would you set a higher temperature for the root LLM calls and a lower temperature for LLM calls deeper in the recursion?

Just wanted to say that I really like this question. Very thought-provoking :)

EDIT: makes me think of many computation systems in various substrates, and how they work. Focus vs distraction/creativity. ADHD workers in hierarchies of capitalism, purpose of breadth vs depth of exploration at various levels of the stack, who's at the "top" and why, etc etc

This is what Codex is doing. The LM has been trained to work well with the kinds of tools that a solid developer would use to navigate and search around a code repository and then to reason about what it finds. It’s also really competent at breaking down a task into steps. But I think the real magic - watching this thing for at least 40 of the last 50 working hours - is how it uses command line tools to dig through code quickly and accurately.

It’s not relying on the LM context much. You can generally code away for an hour before you run out of context and have to run a compression step or just start fresh.

My existing project is very similar to this with some other goodies. I agree with the author that focus on systems versus LLM's is the proper next move. Orchestrating systems that manage multiple different llms and other scripts together can accomplish a lot more then a simple ping pong type of behavior. Though I suspect most people who work with agentic solutions are already quite aware of this. What most in that space haven't cracked yet is the dynamic self modifying and improving system, that should be the ultimate goal for these types of systems.

I read the article, and I'm struggling to see what ideas it brings beyond CodeAct (tool use is python) or the "task" tool in Claude code (spinning off sub-agents to preserve context).

IMO the author is a little over-claiming this work by naming 'recursive'. Quote from this blog:

> Lastly, in our experiments we only consider a recursive depth of 1 — i.e. the root LM can only call LMs, not other RLMs.

> but we felt that for most modern “long context” benchmarks, a recursive depth of 1 was sufficient to handle most problems.

I don't think a size 2 call stack algorithm should be regarded as 'recursive'.

in today's news: MIT researchers found out about AI agents and rebranded it as RLM for karma.

or: found out about RNNs with extra steps.

> Lastly, in our experiments we only consider a recursive depth of 1 — i.e. the root LM can only call LMs, not other RLMs. It is a relatively easy change to allow the REPL environment to call RLMs instead of LMs, but we felt that for most modern “long context” benchmarks, a recursive depth of 1 was sufficient to handle most problems. However, for future work and investigation into RLMs, enabling larger recursive depth will naturally lead to stronger and more interesting systems.

It feels a little disingenuous to call it a Recursive Language Model when the recursive depth of the study was only 1.

I’m not sure if I understood this correctly:

1.Recursion is used to break down the large context and dispatch to different LLM calls to get the useful context.

2.This may lead to longer test-time execution on large contexts (even with parallelism in deep recursion), and the monetary cost may increase rapidly.

I think it’s a different idea from using RAG or manually maintaining a context window

correct me if I'm wrong

Hopefully this can solve the problem of Claude needing to compact itself every 10 minutes, blocking execution. It would be better if it was always compacting in the background. But that requires perhaps more compute than is realistic.

Tell it to use subagents more. I often say something like "you're banned from taking direct actions, use subagents for everything" and it can run easily for 60-90 minutes before a compaction.

For that issue, try Codex until Claude catches up to your style.

this doesn't appear to bring anything new to the table.

please correct me if I'm wrong..this is just subagent architecture?

a model calls another (not self) model, which in turn returns without calling anything else. What you'ce discovered is called a function call.

It simply hopes two drunks are more coherent than one drunk.

Extending this so that the Root LLM can choose the best option from many other LLMs seems pretty powerful.

This isn't just context optimization. Not much different from agent-to-agent workflow IMO.

https://arxiv.org/abs/2510.04871 another recursive based model

> TRM obtains 45% test-accuracy on ARC-AGI-1 and 8% on ARC-AGI-2, higher than most LLMs (e.g., Deepseek R1, o3-mini, Gemini 2.5 Pro) with less than 0.01% of the parameters.

It's a completely different kind of recursion for a completely different (non-language) task.

I actually came here expecting this to be a language model application of that recursive reasoning paper.

Recursion is so popular in computing that this term “recursive language model” is heavily overloaded

It was even before the rise of LLMs

The authors may want to consider a more specific name

It broke new ground!

Reply on news.ycombinator.com