You can have a stable/growing population with only 1 male per n>1 females. So, the evolutionarily optimal strategy is to crank up trait variance in males and have females select only the top 1/n males (hand-wavily speaking). Most modern human cultures suppressed this dynamic via socially enforced pair bonding. When you lose that social adaptation, you revert back to more atavistic matching dynamics.