Improving Language Models by Retrieving from Trillions of Tokens

It sounds like after the model is trained, it also pulls from the training database at generation time. Can someone explain how this isn't a kind of "cheating" at generation?

Happy to be wrong.