A piece of data on its own is usually pretty harmless, a gigantic database that is organized, categorized, and distributed, is potentially very harmful. And it's a smooth gradient between them.
An example of data that is harmless on its own but harmful at scale is license plate reading. https://arstechnica.com/tech-policy/2015/03/we-know-where-yo... a few points tells you that a person is in a city, a few more tells you where they live, a few more tells you where they work, a few more tells you where the shop, a few more tells you who their friends are, who their associates are, if they're seeing someone on the side, if their going to any gatherings, if they tend to frequent red light districts... ect.
In this way, as data is aggregated, the picture of a given person slides from blurry to clear. And a clear picture of someone is an invasive and dangerous picture of someone.
As a result, aggregation of "public" data like this is something privacy advocates generally resist. It's not that there's is some line that's been crossed.