To some, though, it may very well be true. But that's an issue with today's slogan- and owning-the-other-side- based discourse. For what it's worth, I agree with you that merits alone should make the hiring decision.
As someone you'd probably label "woke," I think we should also acknowledge that it is entirely possible and common that inherent bias makes that difficult for white people in hiring positions. Yes, that can happen with ageism as well.