Hacker News Reader

> It seems to work well, mainly hurts when the problems are invisible, so security, backups, breaking migrations, perf, maintenance over time (language / runtime updates, dep updates, server updates etc). All the stuff that hurts if you have never been responsible for the uptime of something before. Not knowing that software is never "done". But junior devs trip on those exact same things.

If we continue from your "water comes from the tap" example this situation feels like someone who has no experience of plumbing deciding they're going to replace their bathroom without bothering to find out how any of it works. They'll possibly end up with a bathroom that superficially works but they're also liable to find problems down the line with leaks, things not draining properly, or water pressure issues.

Some of the people writing software purely via LLMs are even turning this into a business, which I guess would be analogous to deciding you're going to run sewage treatment for a small city without understanding anything about how that works. It'll mainly hurt when the problems are invisible, so food poisoning, or inlets backing up and overflowing into a nearby river.

That analogy breaks down immediately because it's missing the critical novel component: the AI.

In the bathroom remodel analogy, the AI is the plumber that you hire to do the work for you. You tell it what you want, it figures out the details necessary to get it to you. If what you want isn't possible, or requires large changes, a good plumber will raise the issue to you and explain the constraints they are working under, ask for guidance, and offer recommendations.

Sometimes the AI does things in a weird or inefficient way, or neglects details that become important later, or misunderstands something important about the task resulting in an outcome other than what you wanted, but that's also the same as the plumber. This is why people usually want to be at home and keep an eye on the people working on their improvement projects or repairs, because they want to make sure that the guy they hired is replacing the broken faucet, not the functional but odd-looking one.

There is still value in knowing the underlying systems well enough to present a good design vision to the plumber up-front. It's still possible to have a competent executor produce a bad idea, which is how we get stuff like Groverhaus, the Doom Bathroom, and McMansion Hell, which are examples of "slop architecture" in exactly the same way as we are seeing "slop software".

There were terrible software developers before LLMs. Honestly for a lot of people AI generated slop is a massive improvement over what they used to ship.

I agree it's pretty bloody cheeky to run an agency like that though. The only ways out I can see are licensing or real liability.

But let he who has never done dodgy home improvements cast the first stone.

> Honestly for a lot of people AI generated slop is a massive improvement over what they used to ship.

Citation very much needed. Otherwise you're basically arguing that the problems associated with people writing bad code somehow go away when they're suddenly in a position to direct the quality of code generated for them.

I am claiming something slightly different! There are varying effects. I can't really cite, I only have anecdotes.

For "terrible programmers" (I use this loosely, sometimes thats me! - not a value judgement of the person. Might be inexperience, working outside usual area, bad week etc) code shifts from being "fractally wrong how do I even explain to you why it's wrong, you clearly have zero mental model of whats going on here", to "actually ok code but wrong approach and you didn't need to add a 200 line Python script for this". The rework process is explaining the right approach which gets meat proxied to Claude who DOES get it.

I get that a lot of people feel mugged giving feedback and having it proxied, without a _relationship_ where you can see a person is growing and mistakes won't recur. It's like the "author" is just using a robot to steal your time. Sometimes I personally feel relief at work though, because it would steal WAY MORE time to explain detailed and intricate stuff across a really large knowledge gap... in which case I am actually grateful that they have Claude as a cognitive prosthesis. At times I am the one who needs the prosthesis.

Even in the absence of good feedback I frequently see stuff go from "not even wrong" to "thats kinda messed up but it'll get the job done".

Other places, the "good (but not highly conscientious) dev -> slop cannon" effect might be dominating. I feel like AI is raising the floor and ceiling of what people can do (a strong senior / expert can whip the AI until the output is actually good) but having particularly weird effects in the middle. Its enabling people to attempt things outside their usual area (devs doing infra, sres doing more dev, backends & frontends doing each other's jobs) and IMHO that's where good feedback really helps.

I think one of the potentially advantageous things about AI is that it acts as a force multiplier for people who already have expertise in a given discipline, while simultaneously raising the floor for everyone else.

People looking for self-contained, spot solutions to localized problems will have much more capacity to solve them locally, while people with in-depth expertise will be able to build systems that are even more reliable, performant, and scalable than what they were building before.

So we'll get the benefits of disintermediation for the basics, and much greater capacity to deal with effectively with complexity for the not-so-basics.

> However, asking questions and improving understanding of the system and architecture is actually faster and easier for them now than it used to be.

One of the things I've built in my current role is tooling to help understand current architecture, and to resolve the differences between things as-documented, as-implemented, and as-actually-behaving. It's used for a few things, including updating documentation, but it's been most impactful when onboarding new engineers. They can simply ask questions about how things work and why, and get explanations and deep dives wherever they need it.

> Is it only "cognitive surrender" if you already know how to code?

Is the article claiming that?

> Not knowing all the details seems more like the default human condition, in the sense that most people don't know exactly where their water comes from or where their waste goes exactly. "Water comes from the tap, food comes from the store".

Sure. The ability to specialize, and on the flip-side not have to care about a lot of things, remains one of the biggest driving forces of human civilization. But in this article, I don't think the person worried about cognitive surrender is the "most people" in relation to food or water. They're the farmer or the water infrastructure engineer! They absolutely for sure do need to know those things. And all around us we see a lure of systems that invite them to surrender.

> Most of the non-technical people I see using LLMs to create software are not engaging in "cognitive surrender" as far as I can tell.

Maybe not. But the might be cognitively surrendering when they use the LLM to summarize the day's news or write a message to a friend, or even plan dinner for the week. I'd say they are. I agree that it's not clear-cut.

> They're evaluating what they can see, i.e, judging results. They build things for their own use. They test them, iterate on them to meet their needs and try to achieve their vision.

No. They have things built, then they test them. This is different. It might be OK, but it's not the same as building.

> This is roughly the same process they followed when they used to outsource to an agency except the loop is much tighter.

Yes. But they wouldn't usually outsource summarizing a letter from grandma, or the political agenda of a candidate/party in an upcoming election. That's surrender.

> It seems to work well, mainly hurts when the problems are invisible, so security, backups, breaking migrations, perf, maintenance over time (language / runtime updates, dep updates, lib updates, server updates etc).

… or ones own abilities deteriorate.

> Tooling and processes that make that stuff more visible would close a lot of the gaps.

Not the problem of one's own skills though.

> I don't think the person worried about cognitive surrender is the "most people" in relation to food or water. They're the farmer or the water infrastructure engineer!

Fair. Maybe I "cognitively surrendered" a long time ago just as a consequence of working in large teams :(

I hit the breaking point of not being able to read all the code well before LLMs came along. Also people would leave and the documentation for their subsystems would be wrong or incomplete, so there were always mysteries to chase down (during an incident, naturally) even in code written by hand.

Gets to the point, you don't exactly read the MRs, you're mainly checking their approach against what you talked about earlier and making sure it doesn't look way more gnarly than it's supposed to.

Working with agents doesn't feel that different.

> No. They have things built, then they test them. This is different. It might be OK, but it's not the same as building.

It does bug me sometimes when people say "yeah we built the deck..." and they got a contractor to do it. But then we do say things like "Nolan made the film" even though he didn't hold a camera or build the sets. To me, one-shotting doesn't feel like building, but engaging in a loop and applying control to get the outcome you want, does. Regardless of the exact toolset.

I don't personally see super hard lines between using a compiler, a 4GL, a sql query, stitching together 10 libraries and a framework I didn't write, using an LLM or even directing a team.

> Not the problem of one's own skills though.

I read somewhere once we can't weld Saturn V engines anymore, or at least not economically. We would also probably have a lot of trouble finding people who could do all the engine calcs the way they did. It doesn't necessarily mean that rocket progress is stalled.

> Yes. But they wouldn't usually outsource summarizing a letter from grandma, or the political agenda of a candidate/party in an upcoming election. That's surrender.

Agree 100% on the letter from Grandma.

> But then we do say things like "Nolan made the film" even though he didn't hold a camera or build the sets.

I do say that, too. But I'm not in the film industry. My understanding is that those who are, don't use words as crude as "made" to describe a role in the making of a film. I believe they say that he wrote, directed or produced a film.

We are not talking about some other field of expertise here, where we are outsiders – the production of food, the supply of water, or the making of movies – we're talking about our own field of expertise.

> My understanding is that those who are, don’t use words as crude as “made” to describe a role in the making of a film.

Oh, no, film insiders use ‘made’ constantly, or even cruder - attribute credit without using a verb at all. Saying it’s a “Nolan film” is super common. Here’s a VFX person saying “Nolan’s CGI”: https://youtu.be/AFLcP-t0-kI (I’m ex-film industry, and still film industry tangential, and found this clip being discussed on a film industry mailing list.)

Film people are far more likely to pay attention to who the director is than non-film people, and it’s very common for film people to talk in terms of directors ‘making’ the film or otherwise casually assign ownership. I think this is especially true for famous directors and good films, and maybe a bit less true for unknown directors and mediocre films.

Somewhat off topic now, but Nolan is fairly famous for being intricately involved with every aspect of the films he directs. He's constantly behind a camera checking out what shots are going to look like, pushing the limits of where you can put an IMAX camera, and coordinating with set builders to do ridiculous stuff like building a reproduction harbour jetty.

> They didn't read the code before and they don't now.

They didn't read code before, they just wrote it? That makes no sense.