We had a whole, massive global financial crisis that was essentially caused by the fact that this does not work for housing because the risks are so heavily correlated: a large chunk of society finds that they cannot afford the downpayments and defaults all at once, which causes house prices to decrease and the economy to collapse, which in turn means that more people default on their mortgages. It's a systemic risk that affects everyone and that the markets alone do not protect against, which is why regulators have cracked down on mortgage lending so strongly.