Yeah definitely. And also be sure to switch all your keys on your keyboard to a hardwood. I prefer mahogany for its bright clicks and depth of response versus the equivalent plastic keycaps.
Those are the people you usually want as customers. And you're making your website painful for them. That's a decision you can make, but don't be surprised if you'll fail with that attitude.
Keep colors and contrast restricted to the range possible by reflective materials (i.e. what can be printed on paper or ever be encountered in nature) and all the issues are immediately gone. You don't have to use grey-on-grey (which is super annoying). Just keep to the range of colors a newspaper will have, or a zebra will have.