The distinction some people try to draw is when a higher proportion of a product's users are nefarious, but that doesn't really work either because who uses something can change over time.

If you have a society where nobody has window blinds or locks on their doors because it's a rural area and there is no one around to invade your privacy then locks will be disproportionately used by neerdowells "with something to hide", and then busybodies will claim that anyone with nothing to hide shouldn't be concealing their private spaces and anyone selling or using any privacy technology should be pressured to stop. Which sustains the status quo through external pressure even if someone does start invading everyone's privacy.

And that's what's been happening on the internet. Surveillance is the default, Cloudflare et al block Tor users as a matter of course and that drives normal people from Tor and similar technologies even though they would otherwise benefit from its use. People are told that it's the dark web where there are criminals and they shouldn't use it -- it being Tor Browser, the thing that keeps ad networks from tracking them across the internet.

Then after dispersing the normal users who would otherwise benefit from using it, people say that it has a lot of nefarious users to justify the continued harassment of anyone who does. But that's just path dependence, and there are parties interested in leading us down the garden path to mass surveillance.