If one is so utterly devoid of empathy as to be unable to pick up when they are making others (caveat: unnecessarily) uncomfortable--which is to say that an uncomfortable conversation about job performance is different in kind from being a shithead sexist--then they should be nowhere near a position of authority in the first place.
This victim-blaming nonsense should end, pronto.