It would be a bad idea for reasons mentioned upthread. It would not be a bad idea because of the reasons you mention.
You are saying "Let's give in to violence and intimidation". You are suggesting that free speech should not be exercised if there is any threat or violence. That is not free speech.