In my opinion that's very handwavey.
The surface for "interpolation" is fairly broad for LLMs but it isn't infinite -- extrapolation still exists!
figuring out what is and isnt in distribution is quite difficult though given that theres so much data passed in, and transformers have some amount of combining it together
theres some really obvious things though, like picking nonsense tokens together, or things outside of the tokenspace entirely
you could also construct a string that is a series of the least likely tokens in a row. you wont get it to output the next token, despite it being quite clear how you did it