I think this misses the point of going to an ivy league. Would Obama have become president if he didn't have multiple ivy league schools in his pedigree? Maybe, but there's no doubt that the relationships he formed during those years drastically changed the trajectory of his life.
OP is arguing that the undergraduate population of ivy league schools is tiny, so even if they admitted 100% minority students, there would still be millions of minority students who are being left out of the "top tier" education that can be received. Sure, that's true, but the fact that 16/45 of our presidents are ivy league alumni can't be ignored. So when a minority student goes to Harvard and ends up becoming president, he/she can end up changing not only policy decisions to improve outcomes for minority constituents, but also changes the narrative when it comes to who can attain political office at the highest level.
So I would argue that there is absolutely a chance that changing the composition of ivy league student bodies can have an effect on American education inequality. Because some of those students will end up using that advantage to attain political office and affect change at a macro level.