The creepy thing is not that Google is listening to you. (They're not. Period.) It's that they can get so good that they can convince a large segment of the population that they _are_ listening to you.
As others have said, correlating your friends' activity with your colocation and other similar habits is enough.