It's not for making the colors nicer to look at for you but to avoid you putting on clothes which doesn't match in colors and look really off-putting to the non-color-blind (e.g. red pants, pink T-shirt).
As far as clothes, I have a "garanimals" system with numbers to match up shirts and pants and optionally ties. Hopefully my wife is merciful and matching my clothes tastefully.
My solution has been to own a lot of grey shirts.