The problem isn't with the ideal scenario, the issue is when systems fail or are used outside of the ideal scenario. Generally this will be in a muzzling and controlling capacity when it comes to surveillance.
Interestingly merely paranoia of surveillance results in self muzzling and self imposed restrictions.
In the end the results are not conducive for societies progress.