My general understanding of the difference here is that, while you're correct that we don't want any of our information- our mail, phone calls, movements, etc.- to be abused and give us problems, the issue here is that law enforcement DOES have a legal and legitimate need to look at all those things sometimes with a warrant, issued by a judge, who has reviewed the evidence collected so far and deemed there to be probable cause.

The situation we fear will come about instead is this one: law enforcement has access to all of our data, uses our data to find probable cause, present this to a judge, and arrest us.

It's upside down- law enforcement is supposed to be limited in what it can do until it 'picks up the scent' of criminal activity, after which it can use an expanded power set. If we give law enforcement a greatly expanded power set to use before it has 'picked up the scent' of criminal activity, we would foolish not to expect all kinds of abuse (especially given the nature of 'big data'-driven methods of 'picking up the scent').