Hacker News Reader

Comment by zozbot234 | Hacker News Reader

Looped transformers are entirely equivalent to running a single pass through a much deeper network with shared/repeated parameters. There's nothing about it that's specifically "reasoning in latent space" any more than a much larger model "reasons in latent space" from the POV of a much smaller one. An objection to looped transformers "reasoning in latent space" is ultimately an objection that applies to any kind of efficient scaling; it's also worth noting that there's been successful work on extracting ex-post natural language descriptions of these internal latent states, so it's not entirely correct to describe them as inherently non-surveyable.

View on HN

> There's nothing about it that's specifically "reasoning in latent space" any more than a much larger model "reasons in latent space" from the POV of a much smaller one.

I certainly think you could say both of those are reasoning in latent space. But of course people can have different definitions of what they consider reasoning. Maybe this is because I think it's hard to pin down what "reasoning" is so I instead alias it to "computation" which both of those examples do.

There are a huge number of citations about looping on this Ouro page [1]. Ouro is an open looped LLM with a very instructive paper at [2] (also containing references to lots of helpful papers). Basically they get these four results from looping:

• Parameter efficiency. 1.4B and 2.6B parameter looping Ouro models match 4B and 8B transformers on most benchmarks.

• Adaptive computation. Looping is trained in a novel way to avoid exiting too early or too late.

• Reasons to use looping. Looping does not increase raw knowledge storage (approximately 2 bits per parameter for looped and non-looped models) but dramatically enhances knowledge manipulation capabilities.

• Improved safety and adherence. They find that safety and alignment of CoT to final output improves as recurrent steps increase.

[1] https://ouro-llm.github.io

[2] https://arxiv.org/pdf/2510.25741

Hmm, but is it not the case a much larger model could potentially reason more in latent space, depending on how it's set up?

I thought a looped transformer is in fact reasoning more in latent space, though "reasoning" in a somewhat different way from the standard reasoning token approach.

I don't think this is true. The way I understand is that there's

1. coarse grained processing

2. fine grained processing

Tokens are coarse grained processing. While I agree that during a token generation, that processing is fine grained.

But Looping is fine grained processing or reasoning and it is not limited to discrete tokens within passes. This is qualitatively different because a lot more "stuff" happens between tokens. You are right that it is similar to a larger model but then even COT is similar to using a larger model.

Reply on news.ycombinator.com