Because we are all those fools. Do you use gmail? If you don't, do you email people who do? Do you use Facebook? Do people talk about you on Facebook, even if you don't have an account? Do you email people who give Facebook their contacts? Is that even something you can find out?
Trust is transitive. If someone close to you along your social graph is being observed, you are being observed indirectly. If many people around you are being observed, you are being observed rather closely.