Hacker News Reader

Comment by meerita | Hacker News Reader

We are just witnessing the last days of sloppy human-written code. We'll reach a point where most bugs will be fixed.

Most AI-written code is still less thorough about security than a human in most ways, aside from things humans are notoriously bad at like UB in C. I can see a world where maybe a more expensive model audits code afterwards, but the past ~3 years were very sloppy, and it's going to get ugly. There's also still uncertainty about who will have access to unrestricted models that are actually competitive.

I agree to a certain extent. The code will keep getting better because the training datasets will be built on solid AI-generated code. Maybe in the future, training datasets will be 100% AI-generated code and 0% human-written code. Companies are already rushing to develop self-training AI systems to achieve AGI.

But the solid ai generated code is still theoretical. Currently they’re being trained on human code and AI generated code with issues.

If there’s one thing we’ve learned about LLMs so far it’s that they don’t just invent new or better solutions generally. Nearly everything they do is a facsimile or is otherwise derivative. It’s rarely more than the sum of its parts and often it’s lesser than if we did it ourselves, it’s just fast and cheap. So this idea that AI is going to suddenly magically improve its own code and just get better… I don’t know, I’m not convinced we’ve seen anything like that yet and we’re just speculating it will happen because “it should,” which just feels akin to the crazy promises these companies have been making for years that are clearly not happening.

I think agents will loop probes internally, a swarm of them, to test everything until there will be no doubt what is best, what is performant, what will produce a bug, what would be insecure until there's nothing else to probe.

Most AI written code is in Rust through which eliminates most of the problems coming up right now.

This fixes specific classes of problems like memory safety. So does writing in any memory-safe language. But there are still vulns in JS, Py, etc code because it's something like screwed up oauth that the lang of course won't catch.

Reply on news.ycombinator.com