We also have laws against threatening violence.

Reality doesn't fit so neatly into these categories that you're trying to construct, where speech is perfectly harmless unless it's direct incitement to violence and then suddenly it's harmful. That might be how the legal system works but it's not how reality works.

Motivating radicals and spewing racism might not be direct incitement to violence, but history shows that it can have significant negative consequences. The causal pathway is usually non-linear and hard to attribute. But, behind many genocides is racial hate speech that's been allowed to fester for years. Behind many lone wolf terrorist attacks is propaganda, even if nobody directly incited it.

I'm not arguing for or against any specific hate speech law here. Just trying to point out there's a grey area that your categorical thinking isn't good at addressing.