My main problem is cross website tracking. I don't really have a problem with a website keeping track of all my visits to that particular website.
I think it works similar in real life as well. You don't mind if a store clerk recognises you coming back to a shop. But you would be kind of creeped out if he also knew where and when you last visited your dentist.