We do live in a meritocracy, which favor those white, middle class, mediocre males you complained about, because they got all the advantage and thus became good about it.
But selecting the best had nothing with fairness or ensuring that everyone is educated.