Punching nazis will also make you popular if people believe you are punching Nazis. That doesn't mean you should do it.

First, of course, what if that person isn't actually a Nazi? Either mistakenly or maliciously labeled Nazi.

Second, for every boost you get punching a Nazi, a Nazi gets the same boost for punching a Jew or a black person. All you have done is justify their action.