It's an ad hominem attempt to deflect attention back at the attacker. Being wrong is hard to deal with, so you change the subject. It is a fairly subconscious reaction.