I wouldn’t cal it a handicap. I think a lot of it comes down to confidence. Women can tell when a guy is not confident in himself / his appearance and that’s a turn off.