I found this tool recently when it was linked from this Computerphile video[1] about "glitch tokens".

tldw:

Certain junk data was thrown out post-tokenization e.g. the /r/counting[2] community data and debug logs from Rocket League

some tokens specific to those contexts stuck around, however, and are now like "a color you've never seen before" as far as GPT-X models are concerned

giving the model one of these "glitch" tokens causes it to kind of freak out and return gibberish or some completely random response because it has not encountered them during training, because they were removed when the data was cleaned.

[1] https://www.youtube.com/watch?v=WO2X3oZEJOA [2] https://reddit.com/r/counting