I occasionally feel compelled to beat the living shit out of someone. This is probably an evolutionary impulse. Those who cross me in the jungle must be eliminated before they eliminate me, etc. But do I go around beating the hell out of people? Not a chance, absolutely never. Why? Because I don't want to live in a society where people solve differences with violence, and I don't feel violence on my part is necessary.
That being said, I definitely "notice" attractive individuals of the opposite sex. I don't harass, stalk, or assault them in any way, but I do notice them. I don't believe this is sexist. However, going up on a stage in front of people and making a joke about "staring at tits" might very well be construed as sexist because it goes well beyond the act of "noticing", and involves a mixed audience who may not all appreciate the concept.