Yes probably. But considering non-deterministic outputs is the nature of the beast with Llms and we're (mostly) engineers here, calling any part of this mundane sounds almost more like fighting words than just observation
Not guaranteed even with the same seed. If you don't perform all operations in exactly the same order, even a simple float32 sum, if batched differently, will result in different final value. This depends on the load factor and how resources are allocated.