The paradox of tolerance is that if you're too tolerant to nazis, they might take over everything and you wind up with less tolerance.
It is NOT a get out of jail free card for "you don't have to be tolerant to people that you label as intolerant, in fact terminate them with prejudice". You need a credible threat of the 'bad people' taking over and limiting tolerance for it to hold together.
Thank you for coming to my ted talk.