Hacker News Reader

Comment by trzy | Hacker News Reader

Aren’t agents bottlenecked by the underlying models? I’ve read that the number of “chain of thought” steps needed is proportional to task complexity. And if each step has the same probability p of success, probability of success is p^n, where n is the number of steps needed (potentially high). At a 99% success rate per step and 5 steps that’s a 95% overall success rate. 90% drops down to 60%. Not sure what the real numbers are but this seems like it could be a problem without significantly more intelligent ML models?

View on HN

Error-checking and recovery is a potential solution here. Not a well understood one, and might still need higher intelligence than we've got, but-

If your math worked out, then humans couldn't work either.

If you have a "super-agent" AI that is capable of recovering a business process from an error state, why not just use that agent in the first place?

You don't need a super agent, you just need two LLM-based systems with errors that aren't too correlated.

How do you "just" accurately evaluate the error state space of an LLM relative to a real business process? Sounds approximately impossible to me.

If you already have the business process robustly defined as code, then the utility of LLM is unclear. The value prop of LLM is in fuzzy business processes like parsing arbitrary helpdesk tickets.

You evaluate it the way we've evaluated production ML for years, with cheap QC layers sampled and checked by more expensive layers (with humans on top.)

LLMs didn't invent stochastic process steps.

Chat gpt has often given me the right answer for code after seeing the error trace resulting from its previous attempt.

I also often correct my own mistakes based on clashes with reality - I don't just become more intelligent the second time.

I would argue that you are! You will not try to clash with reality the same way you did before, provided you “remember” and I believe future agents/models will have this kind of contextual memory continuously being getting baked in to improve..just a thought.

I think you could do this with an open model with overnight tuning on the day's errors. Probably very expensive though. Easier to scoop up all the errors on the internet on the first round of pre-training.

Couldn’t agree more! That’s why also maybe they are raising 100 more billions!..:p

Are there any papers on this? I’d be interested in reading more.

There's a paper by Papadimitriou (from Logicomix fame) and some collaborators that the transformer model is incapable of solving certain simple problems, and if done by Cost, it needs exponentially many steps.

The paper is currently only available at Arxiv (ie not yet peer-reviewed), but given that it is Papadimitriou, I would be inclined to believe the results.

https://arxiv.org/abs/2402.08164

Reply on news.ycombinator.com