Isn't that the logical consequence of having laws that forbid disparate impact on protected classes?
One could also argue that a person's attributes that are not the result of choice should not be used to assess them, because they reflect historical and environmental influences rather than personal ones.