Ask HN: How much are developers trusting LLMs to write prod code? | Hacker News Reader

Ask HN: How much are developers trusting LLMs to write prod code?

2 pointsdevtor4 comments

I've been out of the dev role for a few years. I'd appreciate if you could help me better understand what its actually like out there. How much are you trusting LLMs for writing production code:

1. Chat-esque LLM assistance while mostly hand writing code

2. LLM writes the code and you review line by line

3. Review mainly the high level design and tests, scanning the lower level implementation for stuff that jumps out

4. Relying on LLM generated reviews and inspecting what it highlights

5. Relying on agentic code/PR/fix loops

How long ago did you hit that level of trust, was it tied to a model release?

Are expectations, team norms etc forcing you to use LLMs beyond your trust in it?

Have the industry changes brought more excitement about or apathy towards your work?

View on HN

I am right now on level 5.

This is a problem I've been working on for the last 6 months. Building commercial apps with AI (agentic development), and not just small, nice-to-show apps.

My problem is that when the agents used to tell me "Done", in fact it was not done. It failed to test real live situations; it tested enough to show me it works, in a nice way, I should say. But in reality, it can never be commercialized. To have a commercial solution, first it has to bring value, then you have all these things around the solution that need to work: usability, help documentation, error catching, monitoring, bug fixing (maintenance), security issues, deployment issues, user rights issues, subscription issues, ... There are a lot more than just the "it does the job" thing. The whole machinery around the job has to work.

So coming back to the question: can an AI agent do that? NO, not alone. It needs a whole machinery around the agents to build a commercial solution.

I am on the final touches of my open source project (yes, after 6 months of hard work ;-)); if interested, I can share what I have done. It already built 2 commercial apps used by real companies (one marketing solution, and one commercial land analysis in Texas).

None in our organization are writing code anymore. We use cursor and stay mainly in the Agent window crafting prompts and validating functionality. Even code review has been handed over to code review agents. The amount of code being written nowadays by agents too overwhelming. It is impossible for developers to review all code manually at this point.

My story is but one story and every organization will be different. Pretty much no one is handwriting code. If you made my VS Code window read only, it would not affect my work. Every pull request is reviewed line by line and manually tested for correctness before merging. But no one is physically writing the code.

I'm somewhere between 3 and 5 depending on the ticket. Fable pushed me over the edge on it but also I've a heavily customized pi that I use which let's me go hands off and not lose control. It's all about tightening those validation loops, watching where weird shit happens and trying to figure out if there was something unclear or ambiguous that let it happen.

I was an engineering manager that stepped back into IC so the idea is code going out I haven't personally inspected was already something I had to come to terms with. Many of the problems I was already solving as a manager translate well to LLMs and agents. Tight verification loops, outside review, easy to use test environment. If you put enough validation steps you'll be less likely to have the agents gaslighting you.

It's never been a better time to build things. If you were coding because you like writing pretty code you will probably disagree. I will admit I am very very bored most of the day which is honestly my biggest challenge right now.

Reply on news.ycombinator.com