The lack or determinism is irrelevant. It's trivial to make an LLM deterministic (certianly much easier than to make a human deterministic!), but that wouldn't make it any more trustworthy.
When you are ingesting potential toxins determinism is crucial. You have a random chance of dying. Facts matter.
Humanity is in its "fuck around" phase with LLMs and is going to fly headlong into "find out" soon.