The problem isn't sexism so much as objectification. It's just shitty to treat someone like meat. It's not going to help people feel like part of a team if they're gawked at. If it's men gawking at women, it's going to alienate the women. Like, seriously, can't people just control themselves and treat other people as people and not sexual objects?
As for the sexism part… historically, you know, men are allowed to stare at women and women aren't really allowed to (their traditional role is to be coy, y'know?). It's an action that is typically masculine, and it's an action that reminds women that they're different. That they don't quite belong. Etc. etc. It's just not positive, it's not helpful, it's just base instinct—and one that should probably be overpowered in the workplace.
It's one of those things that is going to be painfully obvious to someone who has experienced discrimination (eg. sexism, racism, homophobia, whatever) in their life, and not a big deal to those that haven't.