Put it this way: would you rather this article didn't exist, or had less detail? Because that's the alternative in many cases.
5,811 karma · joined May 26, 2016
Put it this way: would you rather this article didn't exist, or had less detail? Because that's the alternative in many cases.
I will say, if the problem is purely code you occasionally see "sparks of AGI" where the model will step back and say "hold on, there is a massive simplification we can make here...". This started happening with Fable-class models. But rarely if ever will the model push for a huge simplification at the product level.
Fair point on the contradiction. I don’t have the answers either.
The meta at this point seems to be mostly around "knowing what to ask for". Eg: I suck at prompting for specific design tweaks, because I don't have the design vocabulary to be specific and articulate what I want. It's a lot of trial and error. I hear people who are developing full scale applications without a tech background run into the issue writ large. For now, I have an advantage over these people when building applications. But perhaps that's just one round of RL away from being obsolete too. Supposedly the model will eventually have such good taste in design/architecture/whatever there will be little point in asking for tweaks.
The only thing that seems relatively future-proof is just being able to communicate clearly what you're trying to do. Not how to do it, or even what success looks like. Perhaps an element of it is even as basic as being honest with yourself.
- Occasionally has strange tics around asking for permission for obvious next-steps, implied actions, etc.
- It's very expensive, both in terms of tokens and % usage on subscription plans.
- Relatedly, effort level is unintuitive. Sometimes it seems like higher effort levels are actually cheaper due to not under-thinking and needing to correct work. But other times they are overkill and send the model into rabbitholes.
That said, it's fantastic as a code-reviewer or "hunter seeker". It's better at finding bugs than Fable and "Get this well articulated task done single-mindedly" is an Astra-shaped task.
Opus 5 has issues too, comment-slop, claude-ish, etc.
5.1 on the other hand can seemingly do no wrong. Easy to work with, writes human-level code. Expensive, yes, but even at Low effort it's well worth it.
For me I added some instructions to speak clearly and it helped marginally and that's fine. There will be a new model out in a few weeks where I'm sure they've laser focused on this issue since nobody can shut the fuck up about it. The same thing happened with GPT if anyone can recall the ancient period of 4-6 months ago.
but the revolution is it doesn't take that long. in like 15 minutes you can chat with fable and get to the meat of whatever the issue is with repeated questioning. and then it does the solution for you. so it's not magic but it's still like a 100x speedup.
But who cares? The point is any codebase over a few years old with lots of customers and a big surface area has lots of code, much of it "legacy" from the standpoint of a guy in 2026.
Yes, this is literally what that means.
Why aren't you guiding your LLM usage? Is that what I said - to spam it and not guide anything? Or to have a careful workflow where you agree on design and maximize your human judgement/leverage?
> any state that's not explicitly being tested and verified in QA loops
As opposed to before, when engineers perfectly reasoned about code behavior from first principals and QA was unnecessary?
Even for a junior making $100K, I have a hard time believe their time is worth less than $75/hr or so.
Edit: Fine, "Senior" is not "Average". But naive salary is not the true numerator.