Hacker News Reader

Comment by claykkari | Hacker News Reader

Or perhaps code becomes just another disposable artifact just like compiler-generated machine code, so understanding or maintaining it will be unnecessary.

View on HN

> perhaps code becomes just another disposable artifact just like compiler-generated machine code, so understanding or maintaining it will be unnecessary.

The compiler development teams make sure that the output of the compiler is correct according to the source code input, preserving semantics and issuing errors for incorrect statements.

Who is doing that for LLM prompts?

LLM is kind of a nondeterministic compiler. You'll want to test the hell out of the resulting code, not only to catch flaws in the prompt but also those introduced by the LLM. I'm bearish on programming but extremely bullish on testing (disclaimer: I'm a programmer).

> You'll want to test the hell out of the resulting code, not only to catch flaws in the prompt but also those introduced by the LLM.

Imagine worrying about gcc or the python interpreter that much. No wonder they use the label "reverse centaur" for LLM usage. You must think carefully before coming up with a prompt, then review the code carefully for correctness. Double the stress for no reason. Or is it just going with the vibes and slinging slop?

Testing just proves presence of bug, not absence of bug.

Sure but that applies to traditional software too, including compilers.

No, with given of defined inputs and condition, you can mathematically show a part of section bug-free, i can't remember the detail, but back then we were using mathlab polyspace to prove that.

That means knowing how output looklike with given input is important.

Obviously LLMs are, and will always be harder to "prove" correct than a compiler. However it's the correctness of the resulting software that matters, right? Coincidentally, LLMs can also help a lot with automated testing and static verification, so maybe the resulting SW won't be more bug-ridden than before LLMs were a thing.

The preference in most industries has always been towards producing more code with some bugs rather than less code that's nearly flawless. Given that, I'm pretty confident that soon I'll not even read code anymore and become a tester (and/or req engineer) instead.

I believe that is true, but more of it is still a liability. Good code tends to be simple. Bad code tends to be verbose, and verbose code tends to be bad. As someone who has been extremely prolific both before and after AI in terms of output, and who monitors the output of LLMs (including anthropic’s best models) I know that they are usually generating a bunch of junk. It will do what you ask but it’s largely terrible. Yes, you can mitigate that with a bunch of adversarial agents, but it’s still not great. Every line of code can potentially be a security hole or cause other problems.

Reply on news.ycombinator.com