if that's the reasoning, i don't see how that buys you anything.
i get that some speech can be reprehensible, but in this case it seems that the the pejorative implications of the language are imbued by people's intentions, not the words themselves. so in a few years, ap will be telling me that "people with" is also marginalizing.
howver, i could be one of "the ignorant" on this topic.