Fabricated issues add up. You're no longer just a minority, you're a minority that wants to change which words others use, that wants others to adapt to them, etc. Even when you're just someone who happens to be black/female/whatever and would really like others to shut the hell up about sex or color.
The majority of people don't have a problem - you're creating a problem for them.