It's not only about where you are, it's also about where you are not at any given time.
You might not see any movement in a house... but maybe there's a bouncer asleep in there after working all night.
Much better to see that the family announced they were spending the day at the beach.
I'm less concerned about burglary than I am about big business.
I wonder how long before we start hearing stories like "My health insurance just went up 300% - when I asked why, they said I visit bars on average twice a week, I eat out at higher than average rates at restaurants ranked on the high side of increasing heart risk, I routinely travel at walking pace through high-crime areas, and I've attended 13 events at which illegal drug use has been reported."
It's your location + all the other public information I can potentially find out about you combined with your location. The combination of a lot of separate facts can, together, potentially make people open to risks that aren't immediately obvious.
For example consider what location from foursquare + sexual orientation from facebook + photo from twitter would give to the scum who gay bash. The combination of religion + location + face is another thing some folk wouldn't want to be public.
I hate to think what the National Front would have done with a "gays, muslims & jews near me" app...