Yes. Imagine complaining about not being able to sit at a particular lunch counter, or getting the seat on the bus you want. I mean, dude, there are other seats and better places to eat. Why are you eating at diners anyway?
Free speech is a good idea for game-theoretic reasons. That's it. Free speech lets people fight it out with only the occasional riot and attempted overthrow of the republic. That's better than the alternative.