LLM heads will say “it’s not completely unreliable, it works very often”. That is completely unreliable. You cannot rely on it to work.
Please product people, stop putting LLMs at the core of products that need reliability.
In practice I think it happens often enough, and I remember a blackhat conference talk from around a decade ago where the hacker squatted typoed variants of the domain of a popular facebook game, and caught requests from real end users. Basing his attack on the random chance of bitflips during dns lookups.
Related, but not the video I was referring to
The chance that the flipped bit changes a bit that results in a new valid state and one that does something actually damaging is astronomically small.
Meanwhile LLM errors are common and directly effect the result.
If you don't understand the tolerance of your scenario, then all this talk about LLM unreliability is wasted. You need to spend time understanding your requirements first.