Complete speculation: They make their money by assessing back-links. So they crawl as many websites as possible, as often as possible, to establish some kind of view of who is linking to what. From that, they extrapolate some kind of popularity score, which they can probably correlate with keywords and types of users and searches.

That is what they sell back to users, presumably. I am actually quite interested to see how clear their data / recommendations would be, but haven't had the time to do so yet.