A minority may think they're punching up at someone they consider to be in the majority based on obvious characteristics, and yet if they don't know the individual they may actually be punching down.

Assuming a person's privilege based on race, gender or other immutable characteristics is imprecise and counterproductive.