Here I am still unable to stop myself from doing line-by-line code review. I notice too many things (redundant code, duplicate code, overly complex architecture, etc.) by doing it that I feel it would be careless not to.
Choose something you want to do but don't really care that much about. Be more ambitious than you think you could be, have a high bar on the outcome but purposefully choose to completely ignore the actual code. Make it write tests, dashboards, monitoring - whatever you need other than code to feel confident.
Now is the cheapest time it will ever be to experiment.
Do you think it's better to experiment with high stakes projects? Do you think it'll be easier to avoid defaulting to line by line auditing of a project that is high stakes?
Ah yes. "all the magic the people say work 100% of the time"
> Do you think it's better to experiment with high stakes projects?
This again assumes that people working on high stakes projects only work on one single project.
This again assumes that people finding issues with LLM-generated code have not tried every magic recipe under the sun from people claiming their hair growth elixir is 100% a working solution.
This again assumes that people experiencing something different from the others are inevitably stupid.
etc. etc.
I don't see how this rebutts the implication.
> This again assumes that people working on high stakes projects only work on one single project.
No it doesn't. It's a very straightforward yes or no question.
> This again assumes that people finding issues with LLM-generated code have not tried every magic recipe under the sun from people claiming their hair growth elixir is 100% a working solution.
He's just describing very common failure modes that come from a lack of structure/linting.
You're framing things like this is some grand conspiracy. OP is literally describing flat tires and your rebuttal is "these magical round tires i keep hearing about." It's very strange.
LLMs will dedupe code if you give them structures to find the duplication. It's not magic. I do it every day.
It's not. OP is literally stating a fact that he sees in his work with LLMs. Without even saying what he's working on.
Your (and others') immediate assumptions:
- he only works on one project
- this only project is high stakes
- they are not magicking their project with right magicks. Sorry, not "harnessing agents"
- they lack structure/linting
Ah yes. It's very strange of me to call out these assumptions.
Now, let me step in the OP's shoes as a person who sees the same issues across multiple projects and doesn't believe in magic (and has tried all of them, except the very expensive ones).
Here's the entirety of OP's text, emphasis mine: "Here I am still unable to stop myself from doing line-by-line code review. I notice too many things (redundant code, duplicate code, overly complex architecture, etc.) by doing it that I feel it would be careless not to."
So:
- no amount of "harnessing the coding agent very strongly" will fix those issues.
Also, "harnessing an agent strongly" is as meaningless as "there's some magic that will fix it". Agents will always continue with code duplication and sloppy architecture. It's a fact of reality.
You can run a "swarm of review agents" that will catch some of that, but this is just a) acknowledging the problem, and b) handing off the problem to the same LLMs that produced the problems in the first place and hoping that they will magically fix them
- no amount of linting will help with things like code duplication, redundant code, or overly complex architecture. No linter catches bad architecture. Very few languages have tools to do semantic code analysis and catch duplicate or redundant code.
- no idea what you mean by "structuring". To structure code correctly you need human input because <see issues described>. Any other "structure" is just running a bunch of the same LLMs on top of various markdown files telling them "make no mistakes" in various ways.
I am not kidding or lying that just today I was "structuring" Opus 5.5 to remove duplication from code it was creating as I was watching it. Despite all the "structuring" and "harnessing" etc. And catching previous duplication introduces by Fable and Astra despite all the "structuring" and things like https://github.com/elixir-vibe/ex_dna
Response: Try experimenting in a low stakes environment
From Google: "Exposure therapy is a psychological treatment designed to help people confront their fears, break cycles of avoidance, and reduce anxiety."
It's those who don't do it who'd need a talk..!
And I say this with the bitterness of someone having lost a big part of what made me enjoy this line of work.
But I have no interest in working as a vibecoder, there are many other jobs that I'd enjoy more
However, I suspect there are teams that are making different choices. In the most obvious example, I can do whatever I want on my personal projects (I hope you agree). And if I started a business and hire in programmers, I have some expectation on defining the culture, making hiring decisions based on my preferences, etc.
Whether or not such teams succeed will be a matter of fact, which I do not believe has been decided at this time.
How does "low-stakes environment" fix that?
Last time I chose to vibe code, I did it in a language I didn’t know so that I wasn’t tempted to try and comprehend the choices the robot made. I ended up with the application I wanted.
Caveat: it was code I would never have to touch again, I ran a script to generate the 140,000 static html pages that the rails app provided and then just uploaded them all to R2 buckets (million reads per month on free tier)
No, they are saying "who cares if all those issues with code are true, just don't review it"
> Caveat:
It's not really a caveat. I also have throw-away code. But we're not talking about throw--away code. We're talking about the systemic issues in LLM-produced code, which are completely ignored and the answer is always "who cares, close your eyes and don't review the code".
The classic consideration is, this code will need to be maintained, built on top of… but it’s not like that was ever a solved problem, big projects remain fragile and corporate turnover means no one knows the entire stack. So I see the vibe code out there as just another imperfect way to build barely good enough software.
I’ve also written code I built slowly and intentionally as a piece of art, beautiful code I was proud of. Didn’t make me any money :)
LLM output isn't compiler output. It's a lot of shitty code that contains duplication, subpar algorithms, bad data access patterns etc. We were complaining about awful software before LLMs. Why is it now absolutely okay to write shitty software just because it's LLMs?
Also: literally no one is talking about "writing software as art". We 're talking about this: "many things (redundant code, duplicate code, overly complex architecture, etc.)"
For example, for all the claims of how LLMs are better than humans, or how you can ship gazillion LoC easily, we've yet to see any significant number of actual good software to come out of this slopathon. Even the frontier labs with unrestricted access to their own models who boasting how they merge 30000 PRs per day [1] ship unbelievable buggy shit. But it's okay because it's LLMs who cares?
[1] https://claude.dev/blog/solving-code-review-at-anthropic/
The person is saying "I am still unable to stop" and that is most likely because they feel a sense of responsibility. If they choose a project where there is no responsibility, where it literally doesn't matter if the project fails, they may feel able to stop. And if they manage to make that project work, if they can succeed, then they will build up confidence.
It seems better to be able to work both on projects where they review all code as well as projects where they do not review any code. In fact, keeping their job in the long term may (or may not) require having both skills.
They are not overblown. LLMs do all the things the person described.
> Or they may learn that they cannot build any kind of software without reviewing every line of code.
We already had bad horrendous software before the onslaught of LLM-slop. Why contribute to that?
> And if they manage to make that project work, if they can succeed, then they will build up confidence.
Confidence in what? It doesn't fix the underlying issue. And the underlying issue isn't "I can't stop" but "redundant code, duplicate code, overly complex architecture, etc." that LLMs regularly produce.
> It seems better to be able to work both on projects where they review all code as well as projects where they do not review any code.
"If I close my eyes maybe this goes away"
(You're welcome to make your substantive points thoughtfully, of course.)
Additionally, do you really believe that 'Now is the cheapest time it will ever be to experiment'? Why so? Because of token prices? Not a good omen if so.
I do believe the subsidies offered on Pro accounts driven by the current competition make access to the best coding models and harnesses as cheap as they will ever be.
God I hope you're right. Maybe then this era of vibe slopping out photoshop clones that'll be abandoned within a week (while advertising them all over social media) will come to a close.
First hit is free ;)
Was this with the latest models (Fable/Opus 5.5 or equivalent) or an older model in a 3rd party harness?
Are you in the camp of: "if at first you don't succeed, never ever try again".
This was a few months ago, so obviously not with the latest models, but:
1. That argument is evergreen. You will always be able to make it, and it doesn’t really add much. It is impossible to refute.
2. Gone back to - when writing code - having a strict mental modal and good control over the codebase. A mix of hand coding (to establish the mental modal) and prompting (for boilerplate). It’s not quite as fast, but the quality is much higher when you mix in using LLMs in quality enhancing capacities (review, debug, etc). This is with the latest model. Hands off still atrophies over time (even the boilerplate), and in some cases is very inefficient (as in wasting my time, nothing to be said about tokens).
> Are you in the camp of: "if at first you don't succeed, never ever try again".
This is great in some domains, but “don’t make the same mistake twice” is much more important in mine. I don’t have the luxury to ship low quality, especially when LLMs make even higher quality so much easier.
If you cannot tolerate failure then don't experiment. Don't mix the two.
I am happy that I have a useful tool, but it simply isn't maintainable without a machine or me investing much time to understand it.
I don't see how the former is a sensible future for humanity. I am happy to say I won't be releasing this software I've created (I will find it useful myself), but deferring all understanding to a black box seems a poor future for brain activity.
I mean, I've had useful software that I don't understand how it works where I am stuck hoping some small company continues to maintain it. And I've had some of those companies go out of business and leave me with useless software that I eventually have to replace once it no longer works on my machine.
Having the code that I can dump into an LLM to understand myself or have it alter it seems better than that.
I tried that. ChatGPT's result was laden with loud, clear and wrong fabications which I might have believed had I not examined the code.
Unreliable program be unreliable.
It's the equivalent of promoting a sausage factory to be the solution to all of mankind's food problems. It's garbage.
* The agent will fix a bug when I tell it to, but only in exactly one place. I'll notice that another place has that bug later and have to redo it. It's the sort of thing that I would know to check for if I were fixing it myself. Yes, I can ask it to look elsewhere for similar bugs, but even that's not super reliable.
* The user-facing text it generates is just as awkward as ever. I have to rewrite all of it, and sometimes it slips through. It's also hard to write text when you don't know what the code is actually doing.
* It doesn't have a good model of what kind of information is relevant to humans. It's just as likely to produce a paragraph of random jargon as a clear explanation of what's actually going on. It shows excessive detail for things that don't matter, and not enough detail for things that do. For example, it doesn't understand that $0 can logically mean "null" in terms of how humans think about things (e.g. "This item is free" vs. "This item is $0").
I kind of wish I hadn't gone down this path but now it's far too late, I can't really manually maintain this amount of slop. I'm quite sad if this is going to be the end state of this profession.
I see the same in e.g. simple CSS. I ask ChatGPT to apply styling X to aĺl instances of say a UI control type. Then to another. Later I ask for a tweak of X. Bot updates some but not all instances. It didn't create a single class for all. No intelligence seen.
The most remarkable fails are simpler though. E.g. at version 303 I ask "Revert to version 299". I get 299 with 303's edits. Bot explains it interpreted "Revert" to mean something no intelligent programmer would.
And most importantly, these are just the ones I directly notice. To see the total, I have to read through and understand the most repulsive code I've seen in my life.
The only way to be satisfied with bot code is by having exceptionally low expectations. And never ever let yourself be kidded that your helper is more than a stochastic parrot gorged on stackoverflow etc.
They've built an entire strategy around moving slow and doing it better
And if people are not looking it, those people don't care about their customers.
> those people don't care about their customers
where things like "it tastes good to me" or "cement didn't crumble when i hit it with a hammer" is good enough to ship we'd be inundated with lawsuits and customer claims... its a total joke where the software industry is headed
Code quality matters to AI just as much as it did to people
If you let it go buck wild on a massive change, there are those times where it sort of paints itself into a corner but rewrites are cheap (for now), so it's just not that big of a deal, imho
The worst part about this is that if you aren't watching it, the AI will "solve" this with a thousand assert_3_plus_5_equals_8() test cases that just further clutter the workspace and the context window. Churn baby churn.
AI tends to add new wheels for a new feature in a large codebase, because finding, understanding, and generalizing the existing code to integrate a new feature will likely break existing features, cost more effort (and token) to implement and test. Human reviewers also tend to like these code since they are clean, easy to review, and get the job done, especially when they are not familiar with existing codebase.
The issue is that wheels like these accumulate in the codebase, and they inevitably entangle. When an upstream specification changes, when a new feature that works with many existing features are introduced, all these wheels need to be revisited, and in many cases they are not because the new test cases do not cover all the paths. The code base got larger and more complex, and contains more bugs than the unit tests can cover, especially when unit tests are often "bent" to fit the wrong behavior unnoticed.
Funnily enough, the models learned from GGC and LLVM, yet, cannot produce something half-decent.
Do what you want, write apps with AI and don’t review code, it doesn’t impact me. The claim that no one cares about good code is demonstrably false, and that’s all I’m objecting to.
so long as his software isn't used for anything that can affect you
I do think the capabilities of AI depends a lot on how you work. If you're doing very low abstraction, fail fast defensive programming in something like Rust then it's fairly good at writing code with high quality. If you're working in some OOP language with tons of freedom, then my experience is that it's usually not better (or worse) than most human programmers, but certainly much worse than some.
I've got decades of experience in enterprise development and I've also been an external examiner for CS students. We have people with no coding back ground vibe coding internal tools in Python which are terrible, but not really worse than what we've previously bought from some of the cheaper coding houses around here. At the same time we have AI writing internal SDK's and libraries in Rust which will frankly write better code than I do on a thursday afternoon after too many meetings and too little sleep because the children kept me awake all week.
Good code can mean a lot of things though. If you're deep into clean SOLID DRY OOP programming I suspect that an AI will produce unreadable horrible code, much worse than most humans. On the flip-side the AI probably knows the difference between a generator and looping over a List. It probably also knows more about JVM optimization than most of those banking programmers trying to replace COBOL with JAVA.'
I doubt Antropic is a good sample though. Bun is written in Rust and it's a single purpose software, which seem to be an optimal environment for LLM's.
Similarly, once you pull in terrible AI slop code, the project becomes unmaintainable by humans and you need to maintain it with AI.
This is, of course, tangential to your point.
I do, my coworkers do, my friends do, etc etc.
Your end users certainly don't either, heck they'll gladly sit there and scroll through their phone until the 50GB React app loads.