They'll have to scan it for search. And spam, as sibling comment mentions. And, really, to store and display it at all, depending on your definition of 'scan'.
Searching is not "scanning". Nor is any other normal behavior of providing email services.
So the question at this point is does Google keep your email data silo'd or do they use it for other purposes?
It’s not difficult to put together creepy functionality and twist it so one of the purposes is providing email service.
Pray, how does it work, then?