It is very much a product of racism. Check out the book “The Color of Law” that covers this issue extensively, and even beginning with the Bay Area.

Anecdotally, when my parents bought their house in the east bay, it came with old (now invalid) covenants that literally said no black people could live there unless they were servants. The house was built in the 40s!

A simple Google search on Redlining (https://www.google.com/search?q=new+york+times+redlining) will reveal how current housing laws and massive wealth transfer that interest mortgage deduction is from less fortunate to the wealthy. There is a near unanimity among economists that housing subsidies are a bad policy and exacerbate rent seeking behavior. If not racist, those policies are downright classist (the line can be very blurry), designed to keep a small minority of homeowners (akin to landed gentry) in power.

Lot sizes in suburbs were (are?) intentionally large to keep out "those" people from purchasing property, further reducing supply.