Slavery wasn't and still isn't that long gone.

The reductio ad absurdum where any mention of a color is considered harmful is based in the assumption that the issue is with mentioning color. It might be useful to distinguish when a color is just a color (e.g. red/black trees), and when a metaphor is borrowed from an actual crime against humanity (e.g. master/slave).