Well, Google doesn’t know anything until you go to a Google page either. Not counting things like Google Analytics, of course, but publishers choose to share that info with Google.
Google quoting privacy concerns is especially disingenuous, as they can track you and your activities across multiple devices.