I was just listening to that podcast yesterday and loved the way he presented this information. I know next to nothing about NNs but I could pretty much follow everything that he said (from an intuitive sense, anyway, if not a technical one).
This "magic" of converging networks reminds me of how ensemble methods, such as random forests, are effective but people aren't sure why. There's certainly (AFAIK) no theoretical grounds to say that a bunch of random decision trees should yield universally good results.