The process has exactly zero chance of never terminating in the usual probability space of infinite strings with iid characters, even though the set of strings (of RNG results) on which it does not terminate is nonempty (uncountably infinite, even, provided N ≥ M + 2); the term of art is that it terminates almost surely. An alternative definition is that you can throw out these strings from the space and no well-posed question of probability (as opposed to set) theory will get a different answer. They’re gremlins, essentially, except for the part where an (uncountably) infinite union of gremlin (null) sets may not yield a gremlin set.

(The usual definition of that space via “cylinder sets” may seem contrived, but it’s usually introduced first because it’s “elementary” in that it does not require developing the machinery of limits of [not in] probability spaces. Those can be made to work, though, and then you can say that the space of infinite strings is the limit of the spaces of length-n strings for n → ∞ and obtain the same thing. In fact, the cylinder-set definition is essentially the limit definition with the notion of limit inlined.)

Actually, it doesn't.

If you're working with real numbers, then your step counter is a natural number, you repeat for[2] infinitely many steps, and the probability of failure is exactly zero.

If, on the hand, you're working with surreal numbers - which you would have to be for "infinitely small (but non-zero)" to make sense - then your step counter is a ordinal number[1], not a natural, and you repeat for[2] non-well-foundedly many steps (one for each ordinal), after which the probability of failure is again exactly zero. (Otherwise it would be the reciprocal of some surreal number, that is less than some ordinal, and for any ordinal, you can show that you tried a well-founded number of times that is nonetheless strictly greater than it.[0])

The more existent problem is that you can't put any useful upper bound on how many rerolls you'll need, which is terrible for constant-time algorithms, and particularly for cryptography.

0: And you actually only need two to the power of the number of tries to be greater, since the failure probability of one sample is at most 1/2, so the probability after K tries is at most 1/2^K.

1: ie, a positive whole surreal number, rather than a positive whole real number

2: Technically, you repeat for up to but not including that many, since you're guaranteed to terminate by then.

This is a rather sloppy way of saying "this process may fail to terminate, though the set of nonterminating outcomes has probability 0". No real number is infinitely small but nonzero.

But that probability rapidly drops below the probability that you will drop dead from some random cause while waiting for the roll's result :)

The die also has a very small chance of landing on an edge between two faces, which also requires a repeated roll.

Which doesn't matter on objects or programs because that chance already exists as a baseline. I'm not building a math here.