Hacker News Reader

Comment by ctoth | Hacker News Reader

> other side???

> We don’t have to look at assembly, because a compiler produces the same result every time.

This is technically true in the narrowest possible sense and practically misleading in almost every way that matters. Anyone who's had a bug that only manifests at -O2, or fought undefined behavior in C that two compilers handle differently, or watched MSVC and GCC produce meaningfully different codegen from identical source, or hit a Heisenbug that disappears when you add a printf ... the "deterministic compiler" is doing a LOT of work in that sentence that actual compilers don't deliver on.

Also what's with the "sides" and "camps?" ... why would you not keep your identity small here? Why define yourself as a {pro, anti} AI person so early? So weird!

View on HN

You just described deterministic behavior. Bugs are also deterministic. You don’t get different bugs every time you compile the same code the same way. With LLMs you do.

Re: “other side” - I’m quoting the grandparent’s framing.

GCC is, I imagine, several orders of magnitude mor deterministic than an LLM.

It’s not _more_ deterministic. It’s deterministic, period. The LLMs we use today are simply not.

Build systems may be deterministic in the narrow sense you use, but significant extra effort is required to make them reproducible.

Engineering in the broader sense often deals with managing the outputs of variable systems to get known good outcomes to acceptable tolerances.

Edit: added second paragraph

I'm not using a narrow sense. There is no elasticity here. See https://en.wikipedia.org/wiki/Deterministic_system

> significant extra effort is required to make them reproducible.

Zero extra effort is required. It is reproducible. The same input produces the same output. The "my machine" in "Works on my machine" is an example of input.

> Engineering in the broader sense often deals with managing the outputs of variable systems to get known good outcomes to acceptable tolerances.

You can have unreliable AIs building a thing, with some guidance and self-course-correction. What you can't have is outcomes also verified by unreliable AIs who may be prompt-injected to say "looks good". You can't do unreliable _everything_: planning, execution, verification.

If an AI decided to code an AI-bound implementation, then even tolerance verification could be completely out of whack. Your system could pass today and fail tomorrow. It's layers and layers of moving ground. You have to put the stake down somewhere. For software, I say it has to be code. Otherwise, AI shouldn't build software, it should replace it.

That said, you can build seemingly working things on moving ground, that bring value. It's a brave new world. We're yet to see if we're heading for net gain or net loss.

If we want to get really narrow I'd say real determinism is possible only in abstract systems, to which you'd reply it's just my ignorance of all possible factors involved and hence the incompleteness of the model. To which I'd point of practical limitations involved with that. And that reason, even though it is incorrect and I don't use it in this way, I understand why some people are using the quantifiers more/less with the term "deterministic", probably for the lack of a better construct.

I don't think I'm being pedantic or narrow. Cosmic rays, power spikes, and falling cows can change the course of deterministic software. I'm saying that your "compiler" either has intentionally designed randomness (or "creativity") in it, or it doesn't. Not sure why we're acting like these are more or less deterministic. They are either deterministic or not inside normal operation of a computer.

To be clear: I'm not engaging with your main point about whether LLMs are usable in software engineering or not.

I'm specifically addressing your use of the concept of determinism.

An LLM is a set of matrix multiplies and function applications. The only potentially non-deterministic step is selecting the next token from the final output and that can be done deterministically.

By your strict use of the definition they absolutely can be deterministic.

But that is not actually interesting for the point at hand. The real point has to do with reproducibility, understand ability and tolerances.

3blue1brown has a really nice set of videos on showing how the LLM machinery fits together.

> they absolutely can be deterministic.

They _can_ be deterministic, but they usually _aren't_.

That said, I just tried "make me a haiku" via Gemini 3 Flash with T=0 twice in different sessions, and both times it output the same haiku. It's possible that T=0 enables deterministic mode indeed, and in that case perhaps we can treat it like a compiler.

Reply on news.ycombinator.com