Yeah, excepting the whole NSA issue, the fact that Google was previously caught/accused of war-driving every wifi net they could see with their streetview cars etc.
While the feature may be convenient, that does not preclude the worst case from also being true.
Facebook is convenient, but its pretty well proven now that the data conveniently stored within is harvested and mined by "infiltrators thereof"