They haven't memorized the entire internet let alone the entire world, a few new avenues:

1. Use recorded movies and TV, buy the rights to transcribe (or get access to the scripts) and pair that with still frames from the video

2. Do the same for YouTube videos, harder to get the rights since you're dealing with Google and all the individual creators but bigger catalogue by far

3. If they haven't already, use the contents of Library Genesis or Google Books or just set up your own scanning project

4. Scoop up any scientific papers that they haven't ingested yet

5. Transcribe podcasts, there are millions of them now and many that publish daily content

6. Pay people to wear recording devices, this one's a bit far fetched but I'm sure it's doable with the right incentives and by choosing regions where laws allow for this

7. Non-English sources, basically applicable to everything above and maybe to the sources they've already used if they had excluded them before.