It's very strange that they don't guarantee determinism. We trained a model at Codeium that does have determinism at inference time - it's not super hard if you don't just use random seeds everywhere.

This makes it a lot harder to rely on OpenAI's API if you are doing any sort of caching / evaluation workloads.