Either people have rights or they don't. We don't need the government to pick and choose how individuals express themselves in order to protect others, not because it doesn't make sense, but because there is no way to do it without a Ministry of Truth, which is arguably even worse.
Remember, freedom of speech is freedom from governmental oppression first and foremost, the kind that arises when authoritarians get to tell people what and how to think.