If you use a deterministic sampling strategy for the next token (e.g., always output the token with the highest probability) then a traditional LLM should be deterministic on the same hardware/software stack.
Sure, that would work.
> How would changing the hardware/other software make a difference to what comes out of the model?
Floating point arithmetic is not entirely consistent between different GPUs/TPUs/operating systems.
Yes, and the one thing that was asked about was "deterministic" not "stable to small perturbations in the input.