People are concerned about things that can't even be expressed without getting labeled as fascist or racist. While the concerns may be irrational and without merit, it should be talked and discussed with those people and trying to understand their concerns better. Instead the reaction is "no need to talk with them as they are racists".
And yes, there are many racists out there and they have gotten so much support lately because many people feel heard by them and only them.