That's why it actually is hard.
Plus, what about legitimate purposes of tracking? E.g. journalists tracking the movements of politicians to show they are meeting in secret to plan corrupt activities. Or tracking Ubers to show that the city is allowing way more then the number of permits granted. Or a journalist wanting to better understand traffic patterns.
The line between illegitimate usage and legitimate usage seems really blurry. Hence my question.