To me, it seems to come down to:
1. there are evil people, but
2. those people frequently have more social power than nice people, and
3. the evil people will use their social power to paint nice people as evil (i.e. "bullying.")
If you're defining the laws for a community or society, or the Terms of Use for a piece infrastructure for such a community/society to use—then it behooves you to consider that any "hammers" built into your system will mostly be used by those with power against those without it, regardless of which side is "correct."
So: If you let people speak freely, the powerful will shout down the powerless. But if you let people silence others, then the powerful will silence the powerless.
Morally, it really comes down to a choice of which kind of hammer hurts wronged innocent powerless people the least. (Which can often mean offering no hammer that can truly be used to "deal with" obviously-evil people.)