Yes race is arbitrary but it is easy to catch on to what a society is referring to.
Its the lack of black people that people worry about with regard to Stuyvesant. There is a lot of literature on how this comes to be. Fawn over “Pure meritocracy” if you like, the outcome in NYC wasnt expected and isnt leveraging the potential of its society.