My theoretical explanation for the gap is that women are sensitive to much more than physical appearances when selecting a mate. When men select a mate, they over-index on physical characteristics, relative to women.

So, women will take into account a man's physical appearance, but also his social standing is also very important, plus perceived intelligence and dependability and so on.

When men look at women, physical beauty is paramount. Intelligence and so on is a plus, but not nearly to the extent that it is for women looking at men.

The other part of the explanation is menopause. Humans live long past the age of sexual fertility, which is somewhat unusual in the animal kingdom. Presumably, ancient human social structures made it beneficial to have grandparents around, in a way that does not exist for most other animals. Relative to other species, this puts more evolutionary pressure on human males to seek out females that are of reproductive age, and ignore older women.

The combination of these factors means that female physical appearance comes out as a prime selector in humans, in a way that it doesn't for other animal species. Male physical appearance is also significant, but relatively less important because of other factors, especially a male's status in the social hierarchy.

Anyway, that's my explanation, but I wouldn't hold any of this as a firm belief. Coming up with evolutionary psychological explanations for sex differences is easy. Actually testing them is much harder.