And even there it is debatable - does saying "people with pale skin are better at producing vitamin D in low-light conditions than people with dark skin" make me a white supremacist?
(Or perhaps people need to become more accepting of a distinction between better at x and better. Michael Phelps is certainly better at swimming than I am, but that is mostly orthogonal to whether he is a better person than I am)
Stop making bad imaginary corner cases and stick to the actual problem.
That is precisely the actual problem--the term is loosely defined and thus lends itself to partisan equivocation. For example, lots of people assert that the MAGA hat is a symbol of white supremacy (and indeed it is, albeit only for those people who sincerely believe it to be).