Not only that, when you have a human designed and implemented system you know it's at least coherent enough that someone understood it at some point well enough to take responsibility for it. No such assurance with the llms.
Large company codebases are very much not like that. I've worked on systems (e.g., banking) that have accreted still-running code for as much as 60 years. Nobody understands how it works. Nobody ever understood how the whole worked, and many of the parts were never actually intended to work together.
But I have see quite terrible finance software, patches over patches hold together by a duct tape. Also finance software has to implement large complexity of tax law, which is very large and always changing. It also doesn't help if the company developing finance software sees software development as a cost center and wants always to push the costs down (especially short term costs).
At some point I was tasked with improving the queries/retrieval times and started converting parts of the code and not only was it possible to write tests that would convey the actual meaning of the code and crystallise it but I started finding bugs in the implementation that no-one realized were even bugs because they no longer knew what it should do when queries grew to certain complexity and the search space was in the hundreds of millions of rows that thinking through them just with napkin calcs wasn't cutting it (specially when the bugs were subtle and the results still allowed the program to work "mostly" correctly).
I also don't even know if it's really true. Sure, maybe it's not coherent enough that a person understands it, but if an LLM wrote it, you can be ~99% certain (in my experience) that that LLM will understand it, and usually in less than the time it took to write it. And unlike humans, the LLMs don't die and take their knowledge with them, or change companies, or get ornery when you ask them to explain.
Also, in my experience, Claude doesn't really do what the parent is saying and dive head-first into complex problems and bork your code. It is fairly humble in the face of complexity. It constantly recommends breaking down larger problems into smaller ones that are more easy to grok.
But most importantly, anyone who's a decently skilled software engineer and who's using LLMs to write code is going to be thoughtful. Especially when they're writing code that's important. Unless you're an AI researcher pushing the bounds of what LLMs can do, which let's face it, the vast majority of programmers are not, it's almost always the case that you'll have your LLM writing intelligible code that you can understand and have it explain easily enough.
Sure, you can do that, but irreparable harm might have been already caused. Sure, that’s unlikely when your product is an LLM wrapper. But in something like a database, it could be catastrophic.
(Yes, there will be individual exceptions to this, just like with everything else in life.)
lol, I assume you’re not a parent then. Or at best a new parent.
Is this a definitional argument? As in, if you’re not thoughtful, you must not be a decently skilled engineer? Because I see no reason to believe that people are going to be thoughtful about LLM-generated code. Just the opposite. It’s incredibly tempting to LGTM it.
I think that's true for low value code. But the more important and crucial and high-stakes code is, the more effort people put into it, generally. Even with LLMs.
So the real question would be, how many people designing / implementing the boot process of any given system don't understand what what's going on.
Those sorts of changes are really rough to manage. Maybe still manageable by LLMs but we'll find out.
Also, for what it's worth, LLM research is hardly the first scientific research to have accidents with negative external consequences.
The comment https://github.com/anthropics/claude-code/issues/30407#issue..., written by Claude Code, and posted by the human GitHub account says:
> "Issues here are auto-closed by a bot when they've had no activity for a while and are missing information needed to reproduce (usually a version and steps, or a /feedback ID). Nothing is triaged solely by counting"
In reality all issues in this repo are subject to time-based auto-closing, unless they collect a number of upvotes. See details at https://github.com/anthropics/claude-code/issues/87647.
Therefore, I wonder whether the internal review process described in the Anthropic's blog for internal repos also results in such issues, and whether Anthropic has a internal way to measure and mitigate that. The blog post says:
> "When someone wants to understand a part of the codebase, they ask Claude to write an explainer for exactly that slice. It takes minutes and reflects the code as it is today."
The Claude Code public repo problem shows that LLMs don't always "reflect the code as it is today", and can be a source of additional misunderstanding.
What you are saying is leaning on 2 premises that seem to not hold water 1. Code written by people is always more readable and maintainable than LLM code 2. Code written by an LLM is always less readable and maintainable
Can you really make those statements in all intellectual honesty?
You need to put 80% in context. There's a reason why FizzBuzz lasted as long as it did - lots of developers, even senior devs, are terrible at writing code, subjected to absurd time constraints, or required to color inside the lines of a leaky abstraction. Sometimes all three!
I've seen many situations at my current job where asking an LLM to do a refactoring made things much better - but if I had to do it by hand I'd just give up and move on to more essential work.
30 years working in this industry and I'd say that's the exception rather than the rule. When it comes to large systems.