Hacker News Reader

Entropy is also added via a random seed. The model is only deterministic if you use the same random seed.

I think you're confusing training and inference. During training there are things like initialization, data shuffling and dropout that depend on random numbers. At inference time these don't apply.

Decoding (sampling) uses (pseudo) random numbers. Otherwise same prompt would always give the same response.

Computing entropy generally does not.

See e.g. https://huggingface.co/blog/how-to-generate

Sure - but that's not the output of the model itself, that's the process of (typically) randomly sampling from the output of the model.

Right, sampling from a model, also known as *inference* (for LLM's).

The inference here is perhaps less pure than what you refer to but you're talking to human beings; there's no need for heavy pedantry.

Reply on news.ycombinator.com