Going to Harvard gets you a foot in the door for your first job, and into some social groups that you don’t want to be in anyway because they are generally super lame and full of nobodies who just talk about who elite they are. Far, far more fascinating are those that lived life on “hard mode”.
Another anecdote, I once did a recruiting effort at Harvard and received a resume from someone who got a perfect score on their SATs and then got on a national talk show for it, with a link to YouTube! I didn’t wind up hiring them but I received a follow up email where he declared that I was rejected by him with a list of reasons why my company was beneath him! Unreal that these people actually exist.