I think it's always useful to define exactly what sort of speech you want to suppress when you discuss additional restrictions on free speech. I think the idea of free speech as an absolute right is already a pleasant fiction - there's many exceptions, and adding additional ones doesn't seem like a huge problem.

But I get the impression that people mean "the initial rally in Charlottesville shouldn't be allowed" when they're discussing this, and that doesn't seem realistic with any sort of free speech restrictions. I'm sorry to parrot a Donald Trump argument, but not everyone in that crowd was waving a Nazi flag, and very few were directly calling for violence (I won't go as far as to say they're good people or historical statue enthusiasts). Do you outright ban any sort of racist sentiment? That seems overly broad and potentially capable of being used negatively against minorities.

I think there's two free speech restrictions that I could get behind:

1. Widening the definition of "incitement" to relax the imminent and direct requirements. I don't see why "kill all blacks" should be protected while "kill this black with this gun" shouldn't be.

2. Banning specific iconography that has a long history of being used for hate (the swastika, white hoods, etc.)

2 may have had some marginal utility in Charlottesville (you'd have no Nazi flags, but still plenty of Nazis). I don't think 1 would have made a lot of difference. As far as I know (I'm happy to be corrected on this), most or all of the crowd was not calling for violence, direct or indirect. How do you restrict the speech of people calling for border walls, muslim bans, restrictions on legal and illegal immigration, and other racist but non-violent positions?