Hacker News Reader

Comment by efficax | Hacker News Reader

but they have slowed down. the delta in capability between model releases now is smaller than ever before and i expect it to continue to diminish until we reach some kind of asymptotic maximum, which hopefully will mean turn towards optimization and local model improvements. The current frontier models running at 10x the tok/s would be incredible for my use cases

View on HN

A little over a year ago they were winning their first gold in an elite high school math competition. Have you not seen what they are doing in math now?

The delta isn't really slower, we're just unlocking capability bundles in batches. The capabilities bundles you care about might have mostly been unlocked already, and just need refinement, which is coming slower than the initial capabilities came, but don't discount the breadth of advances being made across all possible tasks.

opus 5.5 is a step change, you arent using the models enough if you think there is a slowdown

I do not see the current training regime of "relentlessly try anything to get the task done" as a true step change in intelligence. This is one of the reasons I suspect capability acquisition is slowing.

A more intelligent model would not default to spaghetti throwing, it would pause to consider the problem fundamentals, the accuracy of assumptions, and ask questions that remove ambiguity. This however does not lend itself to increasing token sales while prices remain inflated from market and brand dominance.

i burn through a $200 claude and $200 codex plan every week. opus 5.5 is not "step change" it's just a bit better, and the benchmarks show this. Honestly with my workflow I cannot really tell the difference between the top models, they're all pretty much the same in capability, just a bit different. You can build software with all of them, and with all of them you have to validate the change thoroughly to not ship stupid bugs

We're not living in the same universe. Look at capability year-over-year. The asymptotic maximum is wishful thinking. Someone needs to step in before this gets wildly out of control.

Year-over-year, yes I would agree with you, but that is not the cycle time of models. And while I share the same hunch as GP, I'm waiting another cycle before putting more confidence behind it.

> Someone needs to step in before this gets wildly out of control. This sounds like a paid bot, lmao.

I wonder how much is actual improvements in the model and how much comes from better harnesses. I'd be willing to bet that the combination of better harnesses + the models being trained around agentic workflows (i.e the harnesses) has made it significantly better in many things, but it's hard to find other improvements imo.

You can see the difference between GPT 3, 4, 5, 6 each year. Same with Gemini and Claude. Look at image and text generation. Code generation. Math ability (Navier Stokes, Quasi-Riemann.)

Man this is so cringe.

Models just solved a Millennium price and you think their progress is slowing down? I will concede that their capabilities vary a lot from one field to another, but still.

Reply on news.ycombinator.com