Hacker News Reader

This LLM as compiler analogy comes up here and there and it’s not hard to poke a hole in it.

If I generate a piece of code with LLM, and verify it to be correct; can you trust the LLM output, with the same prompt, in a new session based on my verification of my code, with the same certainty that you’d trust LLVM output for the same source code in a similar environment, ran independently? If no, why?

View on HN

Yea, maybe one day in the future, "the LLM prompts" will be considered the source code. But a lot has to happen first, namely the same prompt needs to deterministically create the same intermediate code. I wouldn't check in source code that compiles a different binary every time, and we should not treat prompts as source code while they produce different code.

I agree with you on this one; the compiler output remains the same if run the next day, but AI code might not. That is why you need a strong, independent reviewer who reviews the AI code and freezes it after verification, so that next time you do not have to ask the AI to regenerate the code.

Reply on news.ycombinator.com