Yes. If you're discriminating on the basis of gender, how is that not sexist?
When someone self-identifies as 'not a sexist' they also implicitly identify as a liar. To think we can ever get past discrimination -- sex, race, or whatever -- is extremely naive. It is as fundamental to us as making the distinction between 'me' and 'not me'.
Men are better drivers = Sexist
Men are better drivers, so let's not hire women = Sexism
Women are more attractive to me = sexist
Women are more attractive to me, therefore I choose to only date women = sexism
Efforts to counter sexism against women are not sexist against men.