It tracks, actually. Men tend to pick a sexual orientation and be disgusted by the other sexual orientation[0]. Women are a lot more hetero- (and homo-) flexible.
[0] It works both ways! A guy once told me and my wife that we made a cute couple despite how disgusting heterosexuality really was. And I was like... okay, thanks?