The reality is that these homes are unprofitable investments especially in the short-to-medium term. No sane banker wants to make an unprofitable investment. The only way the investment gets made is someone who gets extra intangible utility from the investment (e.g. they like the neighborhood, close to family, etc.), has a longer time horizon, and lots of extra money to support the short-to-medium term financial costs.
It's not a race thing. It's a money thing -- banks and investors will steer a wide berth around people considering clearly unprofitable investments, so those investments will frequently not get made, and when they do, they will tend to be made by people who are willing to put up their own money.