All the words they have experienced up to that point are part of the training set, as well as all the people and things they have seen.
For comparison GPT has like 1TB of text, and they're hundreds of thousands of books, articles, wikipedia, and so on. So already 3 orders of magnitude more.
And of course the "16 hours x 150 words per minute x 3 years" is totally off by a few orders of magnitude itself.