Exactly, this is just going to open up a pandora's box. What about black box testing, is that racist too? And english terms like blacksmith and black-belt?
There are terms like "blue-collar vs white-collar" too which (according to some) deems the physical labor class as inferior to the "white" intellectual class. And there are general english phrases too like "things aren't always black and white, there is a lot of grey area in between".