In my view it's not just connections. The children of wealthy people are also, for the most part, children of educated people. As a result, they know how to navigate the college system and to get the most out of it. There are things about college that I've explicitly told my kids. Someone who doesn't know, or is predisposed to believe that college is worthless, won't ever know what they missed out on.
Also, by the time they hit college, kids are already highly differentiated, and this affects their access to the more fructifying opportunities in college. There's this giant Sorting Hat, and it's called Math.
So I don't deny the wealth effect, but I don't think it's fully explained by connections.