Sure. But this is not the only issue in human experience where nuance or subtlety may be required, and we seem to have navigated many of those other issues okay.
I do not believe this sort of dichotomous thinking is useful.
You can prohibit any speech by calling it obscenity, adult content or spam, but for some reason people are much more willing to argue for special privileges for hate...