Nit: in practice, even at temperature 0, production LLM implementations have some non-determinism. One reason is because many floating point computations are technically non-commutative even when the mathematical operation is, and the order can vary if they are carried out in parallel by the GPU. For example, see: https://www.twosigma.com/articles/a-workaround-for-non-deter...