"Unlimited tolerance must lead to the disappearance of tolerance. If we extend unlimited tolerance even to those who are intolerant, if we are not prepared to defend a tolerant society against the onslaught of the intolerant, then the tolerant will be destroyed, and tolerance with them. — In this formulation, I do not imply, for instance, that we should always suppress the utterance of intolerant philosophies; as long as we can counter them by rational argument and keep them in check by public opinion, suppression would certainly be unwise. But we should claim the right to suppress them if necessary even by force; for it may easily turn out that they are not prepared to meet us on the level of rational argument, but begin by denouncing all argument; they may forbid their followers to listen to rational argument, because it is deceptive, and teach them to answer arguments by the use of their fists or pistols. We should therefore claim, in the name of tolerance, the right not to tolerate the intolerant. We should claim that any movement preaching intolerance places itself outside the law, and we should consider incitement to intolerance and persecution as criminal, in the same way as we should consider incitement to murder, or to kidnapping, or to the revival of the slave trade, as criminal."
He is specifically framing the issue as one where an ideology goes outside of the realm of debate and on to violence. So for instance an ideology that says 'attack migrants' is obviously something that should be suppressed because these people have skipped the whole debate step and instead appointed themselves judge, jury, and executioner. But, by contrast, an ideology that argues for reasons why unchecked migration may be unhealthy for a society and lobbies for according change is something some may consider intolerant, but is quite obviously not what he was referring to. On the other hand, he would certainly have been opposed to Antifa which, though ostensibly fighting against intolerance, have once again appointed themselves judge, jury, and executioner and have 0 interest in debate or discussion or their views.