This line of thinking can be used to justify anything. That’s why it’s important to protect the individual and their rights, even in face of what a majority - who can be unjust - wants. And speech in particular, is so fundamental to the idea of freedom, that it should be almost absolutely protected. A constitutional guarantee of free speech and privacy is critical.
There are cases where people have been charged, fined, and even jailed for "expression", but those are largely limited to cases where folks are promoting violence against specific groups (including hate speech, for example, teaching holocaust denialism, promoting anti-semitism or pro-racist ideologies calling for violence).
There are certainly cases where there has been government over-reach, but that is why we have courts, and in general, the courts in Canada tend towards a more broad interpretation of Freedom of Expression. Are there specific cases in Canada that you can cite where people haven't enjoyed Freedom of Expression (including freedom of speech, which is protected under the broader umbrella of Expression).