> LLMs are non-deterministic

This is not true. Non determinism only arises via sampling, and to a much lesser extent, CUDA kernel launch subtleties.