Ostensibly to you it isn’t “fair” in the way you have described it. But from a different view, your son is very privileged to be in the position he is, I’m sure he will do well wherever he goes for school. You’re in the South Bay bubble that has put so much hype and pressure on these things. Of course, education is important to you as a cultural value and practical matter; but consider that students who are capable can even flourish better in, for example, the community college setting for a few years before transferring (also more financially savvy).
Anyways affirmative action for mostly white people (“legacy admits”) already exists. Removing AA based on race may not result in great outcomes for the underrepresented minorities. Huge disparities exist that cannot be corrected at the university level in entirely; but AA gives the opportunity for URMs to enter these spaces at higher rates than they would at a so called “level” playing field (it is never is the problem).
It’s time to start considering that there is far more to life than what “elite” colleges will have people believe.