Being tall had several competitive advantages in the past. Basically before guns were invented the bigger guy usually could kill smaller guys, and get more food for the family. Quite obvious that women would prefer the big guy, and instincts don't change that quickly.