I'm not sure. Googling for "gpu non-determinism" and reading around the results seems to indicate that GPUs are often not deterministic.

I don't know why, but it is easy to imagine a scenario where non-determinism makes sense. For example the computer could say "I am supposed to do this floating point division. Let's see ... my very exact FP-Division unit is pretty hot at the moment. 82°C. Let's pass this calculation to the slightly less exact FP-Division unit then".