Hacker News Reader

> Their behaviour is deterministic This isn't true for LLM generated code!

Neither are compilers? But you still trust your JIT compiler. Determinism is unimportant, likelihood of correctness is what matters. Plus agents are deterministic, no?

Was about to point that out. Reproducible builds are possible, but - depending on your stack - not easy to achieve.

Even if we accept your premise that the likelihood of correctness of more important I still don't think that the two scenarios are similar. I have seen more incorrect outputs from LLMs in the past week than I have seen incorrect outputs due to compiler bugs in my entire career. And without determinism there is no way to be sure that ratio will ever change.

I'm not sure. That is also my experience, but I'm not completely sure that isn't my fault and the compiler comparison would be totally valid if I just used the tool better. When I actually write fully fleshed out specifications and have the LLM handle the implementation, I rarely encounter bugs.

I don't think I've seen a Sol 5.6 or later model fail at a task with a verifiable outcome.

It's when I try to do too little of the defining that I get LLM bugs. In which case maybe programming hasn't changed that much, LLMs really can be reliable compilers, and we're all just being lazy.

Reply on news.ycombinator.com