As a man, I feel like if I took on kids, I wouldn't have the time to work on what I value and still do a good job of raising them. And that's with a stable, 9 to 5 job at a big company along with side projects.
If I were working at a startup, where every deadline slip significantly affects the viability of the company, I would probably feel like I didn't have time for children and my startup.
It doesn't have to be about sexism. Children are hugely expensive, and take a large chunk out of your life. Not everyone is willing or able to pay that price.