It's a delicate balance like the Ballmer Peak. Too much or too little problems in childhood and you don't get the incredible drive of these types.
With my own 'difficult' childhood all I got was, according to Gabor Maté, a hard time focusing on stuff that I actually care about and a whole load of mediocre attempts.