Don't be scared of these guys. I think calling out fascism/racism is always good. Why? For one it sets a counter-example to the "racism is ok" narrative. It must be opposed publicly.

> never underestimate right-wingers sense of victimhood

Exactly, the right will play victim anyway. Not calling them out or even saying "well, other people are racist too" is arguing on their terms. Looking for racism among other parties to capitulate to the right is not an effective use of our time.