They could, but they'd be dating back to the principles that were prevalent when various species of the Homo genus existed. When Sapiens were a threat to the Neanderthals, the Neanderthals a threat to the habilis etc. They were thought of as threats for similar reasons as you mentioned; one being more privileged than the other, which led to an insecurity.
A lot of factors such as the geography, and the natural resources that companions the land play a huge factor in the well-being of the society. From what you're suggesting, I get the idea that you want that to remain as a given privilege that should only be cherished by the people who were born around it. That a human born in terrible conditions is a threat to the more affluent, and it's resources and security, solely because the latter was born in a better habitat.This doesn't seem very different to me than the ideologies adopted by the tribes and the cavemen that date back to tens of thousands of years.
I hope I'm not coming off too strong with my words. The language is my barrier, and I've been known to go off on a tangent sometimes due to that. I hope you understand.