Consistency has to be a tough problem to solve for a service like this, since randomness in the choice of each token is part of the magic sauce that makes LLMs work.
It is definitely a hard problem. There are ways to ensure consistency, such as using beam search decoding, which is deterministic. But that comes with other tradeoffs regarding answer quality.