Hacker News Reader

> I have not tried public LLMs myself. Do they give reproducible results?

Public LLMs I don't know but images generated using StableDiffusion are, of course, totally deterministic.

There really is no reason a LLM cannot be deterministic and if it isn't: fix it (even if this comes at a tiny performance cost).

If you fix the random number seed virtually all LLMs should be deterministic. However, just 1 token difference in the input could produce a very different output, depending on the sampler, model, etc. So, LLMs can be deterministic, but in practice they are pure alchemy.

> it gives you different answers to the same question

Sometimes answer is wrong and then right.

If it is deterministic, then what if gets "stuck" on wrong answer?

They do that on purpose. The API gives you a setting you can change to 0 or 1 for maximum creativity or maximum reproducibility

Except it's never reproducible. It's a bug probably.

That doesn't work for GPT-4 though. https://news.ycombinator.com/item?id=37006224