This is a ridiculous comparison. A proper analogy would be someone with a swastika symbol on their shirt talking to a Jewish person. I think we can all agree that wouldn't go well.
Jewish people come in all shapes and sizes, and I know quite a few of them. The bulk are very well educated and are able to see history and their own part in it quite clearly. They are definitely not going to like some one with a swastika symbol on their shirt but the chances of that happening here in Europe are relatively slim because there would be a lot of other people - non Jews too - who would take offense.
That's nothing to do with personal involvement but with recognizing that those that wish to engender a second round of Nazism in Europe need to be dealt with before they become a problem.
The question of the usage of the swastika then becomes intent. eg is it used as part of a Pagan, Indian or other non-Nazi design?