In the history of human civilization, before that kind of setup most people didn't worry about having a house, because they lived in villages, so they lived next to their extended family, and either stayed together under one roof or eventually moved couple hundred meters to the side and built their own.
Expectations for quality of life were lower, sure, but the problem of not being able to find a place to live anywhere near your family or job, that is relatively new and getting worse.
> in a handful of countries, for a few skin colours, for a couple of decades. fueled by burning cheap fossil fuels taken from poorer countries
Also: I know it's trendy to make everything about skin colours these days, but that's a distinct USA-ian bias. The rest of the world doesn't work that way.
I won't argue against exploitation of poorer countries as that, indeed, was a big theme in the last century or two. However, note that whatever many hard problems people in those poorer countries have, affordable housing isn't one of them.