I imagine that is how someone who doesn't understand nuance would characterize it.

If you understand why such legal concepts as fraud and defamation exist, you would understand that there is indeed nuance to the concept of freedom of speech.