Maybe the response will be something along the lines of "The Right is so big and covers so many people you can't blame all of them." But you could say the same about the Left. Covers many, many ideologies and sub-ideologies.
I don't deny that there are people on the "left" going against free speech. As somebody born under communism, I have a similar level of disdain for extremists of all colors.
That said, you know that old saying "Your right to swing your arms ends just where the other man's nose begins"? Well, I think this also applies to freedom of speech. No freedom should be absolute, not when it goes in conflict with other human rights that are just as important, like the right for fair treatment and for being innocent until proven guilty. This is why the constitution of many countries, including my own, grants the freedom of speech only for as long as it's not hate speech targeted at groups based on race, ethnicity, nationality or religion. And I happen to agree with that.