Of course, this is what you thought all along. And it's absolutely not true in the US or Canada. But even in places where there isn't an active sect of actual neo-nazis, there is usually a smattering of violent white supremacist organizations who will work with them.

What I suspect you meant is that, for you, it's not really a problem.