You do know "freedom of speech" doesn't apply exclusively to the act of speaking, and "freedom of opinion, not freedom of speech" still includes the act of speaking your mind?

Also, the swastika predates the NSDAP by millennia. It's still not allowed to use it in a context where it means "death to all Jews". Same is true for the slogan "from the river to the sea".