The reduction of women to a collection of body parts is often seen as a form of dislike for holistic femaleness -- it robs a person of their humanity. 3rd wave feminism might disagree with this to a point, but the 1st wave certainly wouldn't.
Basically, it's a question of participation vs. objectification--of being in on the joke vs. being the butt of the joke.
(My point here is the term, not the man.)