All software is broken, but LLMs seem to make the wrong compromises consistently when working in large codebases.
All software is broken, but LLMs seem to make the wrong compromises consistently when working in large codebases.
I agree with that in the pre-LLM world, but I don’t believe it’s the same today.
You can build something with “bad” code that works and is full of technical debt, but today paying this debt is getting cheaper and cheaper.
So if Claude’s wrong design decision leads to having to refactor several chunks of a codebase in order to implement a new feature, does it really matter if it’s Claude who is going to refactor it in a matter of seconds? Especially for a company like Anthropic which has a virtually unlimited number of tokens to spend on it.
Like before AI, that debt is never going to get paid. Because how exactly are you paying off that debt when the means of paying are debt generators?
Interface design is one of the things Claude is worst at in my experience (using it every day for the past year).
Ideally, you design patterns that enable quick replacing of parts that get to messy. The code is cheap, just start from scratch with the defined functionality pre defined instead of bolted on with jank.
My point is more about up-front investment paying dividends. To speculate, it seems like the reward hacking is not there for long-term thinking.
What would they be?