> so that is probably reproducible
Is it? How many more pages get published each day? If you include each YouTube video and social media post, it's easily several orders of magnitude. Organizing this kind of firehose isn't easy, neither is prioritizing/ranking it.
2006 was child's play in comparison.