Hacker News Reader

Comment by zthrowaway | Hacker News Reader

Can definitely attest to this. The frequency of outages at my company have increased drastically the past year, especially ever since incorporating agentic development. I’m seeing all of the dev best practices go out the window. We have a few vibe coders that are posting 15-30 PR’s per day. It’s way too much for us to review. We’re not a big shop. I think we’re going to have to hire more people just to review code across the industry. And those people will have to know how to actually write software otherwise what are they even reviewing. Maybe the models will get so good they never make a mistake. Doubt it.

View on HN

I wonder if the PR workflow is just unsustainable in the agentic era. Rather than review every new feature or bug fix, we would depend on good test coverage, and hold developers accountable for what they ship.

The result might be more faulty code getting merged, but if you already have outages and can't review every PR, is there currently a meaningful benefit to the PR workflow?

Diogenes carrying a lamp, looking for good test coverage

Copy-pasting screenshots of red lines.

This is the "if you're already letting faults through, why not give up trying to stop faults?" approach.

The alternative might be "what if we could get the genie back into the bottle?"

We know some people are using LLMs to evaluate PRs, the only question is who, and how strong the incentive is for them to give up.

This reminds me a bit of monoliths vs microservices. People would see microservices as the next new shiny thing and bring it with them to their next job, or read a great blog post that sounds great in theory, but falls apart in practice. People would see it as as purely architectural decision. But the reality was that you had to have the organizational structure to support that development model or you'd find out that it just doesn't scale the way you expect and introduces its own sets of problems. My experience is that most teams that didn't have large orgs got bogged down by the weight of microservices (or things called "microservices"). It required a lot of tooling and orchestration to manage. But there was this promise that you could easily just rewrite that microservice from scratch or change languages and nobody would notice or care.

LLM-generated code feels the same. Reviewing LLM-generated code when it's in the context of a monolith is more taxing than reviewing it in the context of the microservice; the blast radius is larger and the risk is greater, as you can make decisions around how important that service actually is for system-wide stability with microservices. You can effectively not care for some services, and can go back and iterate or rewrite it several times over. But more importantly, the organizational structures that are needed to support microservice like architectures effectively also feel like the organizational structures that are needed to support LLM-generated codebases effectively; more silo-ing, more ownership, more contract and spec-based communication between teams, etc. Teams might become one person and an agent in that org structure. But communication and responsibilities feel like they're require something similar to what is needed to support microservices...just that services are probably closer in size to what many companies end up building when they try to build microservices.

And then there are majestic monoliths, very well curated monoliths that feel like a monorepo of services with clear design and architecture. If they've been well managed, these are also likely to work well for agents, but still suffer the same cognitive overhead when reviewing their work because organizationally people working on or reviewing code for these projects are often still responsible for more than just a narrow slice, with a lot of overlap with other devs, requiring more eyes and buy-in for each change as a result.

The organizational structures that we have in place for today might be forced to adapt over time, to silo in ways that ownership and responsibility narrow to fit within what we can juggle mentally. Or they'll be forced to slow down an accept the limitations of the organizational structure. Personal projects have been the area that people have had a lot of success with for LLMs, which feels closer to smaller siloed teams. Open-source collaboration with LLM PRs feels like it falls apart for the same cognitive overhead reasons as existing team structures that adopt AI.

> I wonder if the PR workflow is just unsustainable in the agentic era. Rather than review every new feature or bug fix, we would depend on good test coverage, and hold developers accountable for what they ship.

I think what you're describing is setting up the human as the fall guy for the machine.

So taking responsibility for the code you generate is being a "fall guy?"

> So taking responsibility for the code you generate is being a "fall guy?"

Yes, if your boss expects you to use AI agents to generate code faster than you can reasonably understand and review it. You're stuck between a rock and a hard place: you're "responsible," but if you take the time to actually be responsible you'll be reprimanded. The environment pushes you to slack on reviews in the short term to keep your head above water, but when a problem happens because of that you'll be blamed for it.

The proposed industry solution is to use agents to review PRs, as not to slow down the velocity of delivery...

My current workplace is going through a major "realignment" exercise to replace as many testers with agents as humanely possible, which proved to be a challenge when the existing process is not well documented.

So you essentially trust the output of the model from beginning to end? Curious to know what type of application you're building where you can safely do that.

Edit: to clarify, I know these models have gotten significantly better. The output is pretty incredible sometimes, but trusting it end to end like that just seems super risky still.

I guarantee you it's nothing quantifiable.

LLMs can't be responsible for deciding what code you use because they have no skin in the game. They don't even have skin.

If you type fast, well then it takes just as long to code it yourself as review it. Plus you actually get flow time when you're coding.

For heaven's sake people have the robot write your unit tests and dashboards, not your production code. Otherwise delete yourself.

"Hey Claude, did Claude do a good job?"

I did an experiment today, where I had a new Claude agent review the work of a former Claude agent - both Opus 4.6 - on a large refactor on a 16k LOC project. I had it address all issues it found, then I cleared context, and repeated. Rinse and repeat. It took 4 iterations before it approached nitpicking. The fact that each agent found new, legitimate problems that the last one had missed was concerning to me. Why can’t it find all of them at once?

You're expecting it to be a person. It's not.

It is more like a wiggly search engine. You give it a (wiggly) query and a (wiggly) corpus, and it returns a (wiggly) output.

If you are looking for a wiggly sort of thing 'MAKE Y WITH NO BUGS' or 'THE BUGS IN Y', it can be kinda useful. But thinking of it as a person because it vaguely communicates like a person will get you into problems because it's not.

You can try to paper over it with some agent harness or whatever, but you are really making a slightly more complex wiggly query that handles some of the deficiency space of the more basic wiggly query: "MAKE Y WITH NO ISSUES -> FIND ISSUES -> FIX ISSUE Z IN Y -> ...".

OK well what is an issue? _You_ are a person (presumably) and can judge whether something is a bug or a nitpick or _something you care about_ or not. Ultimately, this is the grounding that the LLM lacks and you do not. You have an idea about what you care about. What you care about has to be part of the wiggly query, or the wiggly search engine will not return the wiggly output you are looking for.

You cannot phrase a wiggly query referencing unavailable information (well, you can, but it's pointless). The following query is not possible to phrase in a way an LLM can satisfy (and this is the exact answer to your question):

- "Make what I want."

What you want is too complicated, and too hard, and too unknown. Getting what you are looking for reduces to: query for an approximation of what I want, repeating until I decide it no longer surfaces what I want. This depends on an accurate conception of what you want, so only you can do it.

If you remove yourself from the critical path, the output will not be what you want. Expressing what you want precisely enough to ground a wiggly search would just be something like code, and obviates the need for wiggly searching in the first place.

The fact that anyone in leadership would ever think this is even remotely possible - given my experience in the general state of requirements / contracts / integrations / support - makes me bleed from my earholes just a little bit.

It's starting to just feel a little like an excuse to call everyone on deck for "a few weeks trying 9-9-6". But even then the lack of traction isn't between the eyeballs and the deployment. You'll still be spinning wheels in that slippery stuff between what a customer is thinking and what the iron they bought is doing.

People pushing dozens of PRs per day need to learn to prioritize tasks, and balance a bit more towards quality over quantity.

And maybe spend some time doing reviews for other developers. And if they aren't qualified to be, then maybe spend that time becoming qualified rather than pumping out more slop.

This is the way. There's nothing inherently wrong with using AI as long as it's used responsibly.

I highly doubt there are any managers or executives who care how AI is precisely used as long as there are positive results. I would argue that this is indeed an engineering problem, not an upper management one.

What's missing is a realistic discussion about this problem online. We instead see insanely reckless people bragging about how fast they drove their pile of shit startup directly into the ground, or people in denial loudly banging drums to resist all forms of AI.

Sounds like people need to speak up to management

Management likes how fast features are getting deployed so they essentially told us to just deal with it.

I mean speak up to management in a way where they know it's stupid and they're stupid for pushing it

Management doesn’t care. This sort of thing is becoming more common at my workplace too. More outages, more embarrassing bugs, even bugs that leak customer data. The solution is always more AI, and if you’re still shipping bugs and causing outages, it’s because you did’t use the AI correctly. Leadership makes all the right noises about quality and ownership, but when it comes down to it, the incentive structures clearly prioritize shipping things faster, all else be damned.

Sounds like a fast track to sinking their company into the ground

Management wants to get rid of people; they want to have their "wish-machine" that does what they say without any need to deal with nerds or ethical issues.

Maybe it’s time to have multiple agents and models review the PRs and also provide context for easier human review. That and lots more focus on robust testing.

There’s no way velocity will decrease now that upper management is obsessed with AI.

I really think that software in general is getting buggier, with ChatGPT/Claude being some of the buggiest software I use. I constantly run into quality issues there and I've reported at least a dozen bugs to ChatGPT this year. One kicker I found recently was that Codex PR Reviews, once turned on for a repo, cannot be turned off - I got escalated to engineering who confirmed that they forgot to add a feature to disable code reviews.

Honestly by the time it gets to review it should be rock solid, so the only thing the reviewer has to think about is the big picture and never “does this actually do what it’s supposed to without abusing any of the interfaces”. Vibe coding makes solid validation, testing, and documentation trivial. The onus of proving your code is good needs to shift downward, not upward. And straight up vibing it is absolutely a terrible idea for anything other than a demo or a simple tool.

Reply on news.ycombinator.com