I think for those two examples you listed, the modifier is orthogonal. While "black" or "cancer survivor" tell me more about them as a person, they don't help understand gender/sex. Gender and sex, or the residency words I listed, provide different information about the same attribute.

Saying someone is a cancer surviving resident doesn't add anything to my knowledge of their medical history or which laws apply to them, but resident vs ex-pat tells me a lot.

Where do you think black babies come from? Do you not think black people are human?