Shouldn't display drivers have a "color blind" mode that would alter colors to make them perceptible? It seems an overkill to ask for the whole web to change.
I am always going to be using a computer with normal colors, as are my cohorts. You can accommodate us or not, the burden is on us. But that means we are going to think your website looks bad and/or maybe isn't even worth our time if we have to muck around with tools to be able to use it.
Heres an example of what it does: http://imgur.com/a/giSxg and how bad it can look: http://imgur.com/wlWgC The top is normal, the rest are transformations I'll use (the 2nd image is the one I usually need).
Aside from that, some of the design lessons and considerations that come from considering color-blind users tend to be good design principles in general.So - they may be good considerations to make anyways.