Hacker News Reader

Comment by zamadatix | Hacker News Reader

90% vs 93% is a big deal IMO, especially when you're trying to feed outputs into inputs.

I think the point being made is that when you're feeding outputs into inputs, both 93% and 90% rapidly collapse to 0% without some way to recover.

Which is precisely what I'm responding about above. 93% actually collapses much slower than 90% since the rate of reduction is based on that of all of the previous rounds, not just the current chance. E.g., this is the difference between 14 rounds before an expected failure instead of 10. An extra 4 rounds deep at the same error rate is a big step forward despite a small change in the base percentages. Add just few more percentage points again and you've more than doubled the depth.

I.e. it's precisely when you look at these chained that the difference is actually very large. It's really only the 1st turn or two of prompts where the difference seems negligable.

How are you running that math and what are you consider to be "expected failure"? If even 30% of your runs go into the garbage, that sounds horrifically unreliable, and both 7 and 10% failure rates hit those margins in less than 5 iterations.

Not to mention that the deeper your iterations, the more money burned on garbage before you realize that data in turned to garbage out turned to garbage in somewhere in the chain.

I'm taking 0.90^n and 0.93^n and seeing 0.93 does more than the initial 3% better over time, which preciesly saves you from things like spending tokens on garbage by this larger factor. You're free to pick whatever target reliabilities from that. E.g. you can maintain above 80% accuracy over 3 turns rather than 2 or 50% over 6 turns instead of 3 or you can pick another percentage point and see how the depth to expected error/expected error at depth numbers are much better than 3% and how another 3% actually doubles accuracy because this is all non-linear.

There's also the matter of "90% compared to what". If it was 90% per attempt compared to a calculator then I'd just use a calculator. If it was 90% per attempt for something I have a 10% chance of getting right per attempt then it's not really so bad. OTOH if it's 90% per attempt at keeping my car in the lane then it's obviously quite bad. I.e. there isn't one "reliable percentage" to target which is why I'm comparing the curve instead of just the most stringent things I can think of in isolation.

Completely agree that error tolerance depends on what it is that we're doing. Some things demand no errors at all, some things have a cost to attempt, some things you can't pre-evaluate for correctness if you don't already have the prerequisite knowledge and you only get one chance. Not everything is software where we can conjure 50 more end-to-end tests to ward off demons. When it's not a harmless throwaway weekend project, 7% is already a huge error rate before you ever think of feeding it back into itself.

I of course agree not everything is fine with 93%, but via the very same reasons believe a 3% error rate change is a big deal rather than a reason to dismiss the improvement: not every task has the same tolerance, but there's not many percentage points left until it'd be claiming all of them.

That's completely orthogonal to whether you can still find tasks which require more accuracy or not (except for the sole case someone claims it's 100%).

You might be thinking along the lines of the O-ring theory. Indeed long supply chains accumulate errors exponentially.

However the origins of theory explored the outcome of disastrous and irreparable outcome of space shuttle exploding, lives lost, and billion of dollars destroyed.

When LLM goes off the rails the cost is much lower, and retry is on the table. In fact our core competency shifted from writing code to creating environment where agents can retry cheaply and course correct based on feedback.

A modest increase in the number of turns it takes to reach decorehenve could make a difference between Decoherence and reaching the a grounded checkpoint.

If failure is essentially guaranteed somewhere in that chain, success becomes impossible. You don't need to lose lives to have simply have a fruitless endeavor. And when we're talking about law, we're talking about more than a fruitless endeavor. Law also isn't software that can have tests crafted and fed back into the loop.

First off, the 90% to 93% success rates are not step-by-step failure rates between successive steps of pursuing a solution. They are overall end-to-end success rates on extremely difficult problem sets that most humans would score close to zero on, and most humans with sufficient training and experience would score significantly less than 90%. And they are problem sets that grow successively harder as AIs get successively more capable. A few years ago AIs were being tested on high-school level math problems. Now they are being tested on problem sets that challenge people with PhDs. Or problem sets that humans are (so far) incapable of solving.

Compare that to humans. How often do you think humans get it right the first time? Almost never, for sufficiently complex problems. And how often do humans get the overall result wrong, without realizing it? Also, almost always for sufficiently complex problems, or sufficiently ordinary people. And what percent of the general public would be able to get even one question right in a problem set based on Math Olympiad problems? What percentage of mathematicians are capable of solving even one Millenium Problem?

Furthermore, you have to take into account that AIs do not particularly use output to input architectures. I frequently see self-corrections to plans when I'm developing using AI assistants, and plan changes are sometimes major. ("Oh I see. I need to revise step 4 of the plan." -- Claude paraphrased). Clearly current-generation AIs use more than input to output feedback; they also use supervisor/client strategies of AI and collaborative networks as well. And often provide self-correcting infrastructure, varying from "Claude: Oh. That generated a compiler error. Let's fix that.", to "Claude: One of the test cases is failing. Let's see what's going on.". And that self correcting infrastructure allows AIs to try again and again until they succeed. (Not unlike what humans do when solving difficult problems).

As a member of the 0.2% (but not the 0.00006%), I have to imagine that frontier models, when applied to research topics, are using most of those 10,000 instances to investigate alternate hypotheses in parallel. Which is also something that humans do across entire disciplines. Based on my personal experience with difficult technical and commercial and scientific problems, I sincerely believe that most breakthroughs are not made because whoever made them is more brilliant than the other hundred or thousand or dozen qualified people who are working on the same problem, but are made because they happened to be lucky enough to start with a somewhat random hypothesis that luckily turned out to be the right one. In case you were concerned that this is a sour grapes point of view, yes, our team was the one that came in first.

So even 90% may be a very big deal, for problems that human get right about 0.1% of the time. Or even for problems that AIs get right about 0.1% (but have enough instances available to pursue 10,000 hypotheses in parallel).

p.s. Ironically, it really looks like humans score significantly less than 90% on the "is 90% good" problem.

Reply on news.ycombinator.com