A very problematic tendency with a free housing market and free reign for development is that it ties the price of buildings to the price of the land they sit on. No one is going to, of their own accord, buy a piece of land in downtown Manhattan and build a low-income housing building there. So, free development does allow prices to overall go down even as the city grows larger, but at the cost of moving poor and middle-class people towards the periphery of the city.
Land near the city center and other landmarks is naturally scarce, there is no way to create an abundance of physiccal space. That inevitably drives the price of that land up as the city expands. The only way to prevent the rich buying it all up and excluding everyone else from those areas is for the city to regulate the need for low-income housing being built or regulate the price itself.
A better policy overall, though no one has managed it to any extent, would be to find a way of developing in a less centralized way - stop growing cities to this mega metropolis level, and somehow encourage the creation of more big-but-not-huge cities. Or, find a new way of organizing cities so that they are less centralized themselves.
Abig part of this, I suspect, would be reducing centralization of companies as well. The huge behemoths we have today control a vast amount of resources, so their decisions of locations naturally force people to gather in the same city, and create high value space. If instead you had a much larger range of much smaller companies, there would be less need to have huge HQs with thousands of workers in the same building a la Apple's Infinity Loop 1.
The best proof, I think, that this type of decentralized development can work at some scale lies in Europe. Even though individual European countries and cities have similar problems and tendencies to those in the USA, these are much more recent problems, and overall there are many times more large cities across the EU, and no mega metropolises like NYC or LA. And, many of these cities have city centers that are somewhat frozen in time for 100 or 200 years, not since the 70s, without reaching the ridiculous prices of San Francisco for example (though they might get there).