Hacker News Reader

Comment by gnunicorn | Hacker News Reader

Interestingly, despite it being much more detailed and a lot more process and procedure than what I currently do - which is more akin to the version 0 described, but in parallel - we come up at the same final problem: reviews and quality assurance.

I sign off the code I merged, part of company policy but also just to be sure it is actually decent. But reviewing has become the real draining bottleneck: even stacked PRs, if that total 5-6k lines is not a 5min job. Even if I brainstormed and set the plan, that's really the part that doesn't scale right now for me in this. But the author is very shy about that: either the changes arent that big in the end or they trust the process enough to review in a more casual manner. Being equally untrusting I can't do that ...

View on HN

For decades, engineers understood that large code reviews are harder than small ones. Out of both politeness and a desire to receive better code reviews, we learned to break our large changes into smaller chunks. Some engineers took things even further and replaced code reviews with pair programming. But then LLMs showed up and everyone seems to have forgotten those lessons.

They can be still be applied now using coding agents, if you're willing to push back against the default setup and change your mode of thinking a little bit. Of course it doesn't help that an entire industry is dedicated to persuading us that maximizing token spend is the only way to get shit done.

I appreciate this probably seems like an extremist take, but I wrote some more about it here in case there's anybody out there who identifies with it:

https://philbooth.me/blog/agentic-coding-and-mental-models

Agree with this completely. This push for more autonomy I think is the complete wrong direction for how to use LLMs.

I want less code to maintain not more that I don't even fully understand.

I think research and very supervised coding with lots of guardrails is the way to actually gain productivity from these tools.

I think that's reasonable. My only gripe is that making small sets of changes is often faster to do by hand than waiting on llm reasoning, so I've found it amounts to very little speedup.

> They can be still be applied now using coding agents, if you're willing to push back against the default setup and change your mode of thinking a little bit. Of course it doesn't help that an entire industry is dedicated to persuading us that maximizing token spend is the only way to get shit done.

Yeah the problem is the executives and managers around us are demanding we ship massive features as quickly as possible, and I like having a job and dread having to find a new one in this market...

Give them some time to be slapped with the real AI invoices once Anthropic, OpenAI IPO.

Proper review should take longer than writing it yourself, because you need to know the correct solution, understand the proposed solution, and evaluate the difference between the two. When designing it yourself, you just need to know the correct solution and write it, and with modern high-level languages and IDEs with autocomplete writing it is hardly a bottleneck.

It is harder to solve a sudoku than verify a solution's correctness. I find similar benefits occasionally when coding with LLMs.

Sudoku’s constraints are knownn and easy to build an harness for. Software has a more malleable structure. An harness is hard to build and the tests cases for the constraints can be a lot.

I disagree under the following circumstances, which in my experience is the common case: You don’t know from the outset all relevant considerations that go into implementing something. Coding yourself is an exploration process of those considerations. Being shown a finished solution doesn’t let you see and understand all the considerations and the possible options that you’d have contemplated when implementing it yourself. When reviewing, you still have to do that exploratory thinking to weigh the possible options. And the fact that you have to do that exploration purely mentally rather than in a process of working with code arguably makes it harder (similar to contemplating alternative solutions to a Sudoku purely mentally, actuallu).

There rarely is a single correct way of implementing some requirement or feature. It’s a trade-off between compromises, not binary correct or incorrect like a Sudoku puzzle. The insights that the exploration give you may even lead you to implement something significantly different from what you originally set out to.

Imagine sudoku with hundreds of subtle, sometimes mutually exclusive rules, and no single valid solution.

This is not about LLMs, by the way. It’s about reviewing any code, including by a fellow human. It’s just that many people mistakenly feel like with LLMs they can lower their guard and accept even if they have not gone through the steps of themselves coming up with their solution and comparing it to the one suggested by the LLM.

The reason is that many correctly see proper review as duplicate work, and while it is justified with another human (because it is (A) instructive and (B) reducing bus factor) with LLMs most people simply can’t be bothered. If you personally can, you are a minority.

If I'm attentive during spec/plan creation I sort of build this "expectation" of what the actual PR will look like, the mental model of it. Then it's somewhat easier to review. But the mental load is brutal tbh, and still not sure if it's "worth it"

Reply on news.ycombinator.com