That's a fine argument, as far as it goes, but in practice it's applied spuriously, unevenly, and unfairly.
One of the principals involved in getting Yarvin deplatformed from Strangeloop, Steve Klabnik, is a communist, supporting an ideology that is directly responsible for the murder of 40 million people, and indirectly responsible for another 70 million. Klabnik has spoken, without political interference, at plenty of conferences.
If we were to apply your argument equally, Nazis and Communists would both be deplatformed. But in practice, they are not.