There is a massive gap between top schools and middle schools in terms of the quality of education, the connections you make, and the opportunities you get. The density of smart people is far higher at those top schools he mentioned.
Downplaying that doesn't do anything for anyone.