Perhaps there's too much focus on the truly elite schools, and not enough on the good or very good ones. Surely it's possible to succeed by attending those?
If you had the wherewithal to do well at a lousy high school, you'll be just as able to make something of yourself at a mid-grade university.
Elite universities are important if you want to be in the "elite" clique. If you just want to make a better life for yourself, any old university will get you pretty far. One close to home will likely be a lot more relatable culturally as well. Not everyone wants to uproot their identity and go be someone else somewhere else.