If you ever get the chance to see a talk by Joe Minicozzi from Urban3, I highly recommend it.
He covers this in detail, and his work has been used in a number of Strong Towns posts, like this one:
https://www.strongtowns.org/journal/2017/1/9/the-real-reason...
>To maintain just the roads and drainage systems that have already been built, the family in that median house would need to have their taxes increase by $3,300 per year. That assumes no new roads are built and existing roadways are not widened or substantively improved. That is $3,300 in additional local taxes just to tread water.
I've seen these figures before, and they don't make sense. It looks like he gets that $3300 figure by assuming that payments on infrastructure will double and infrastructure will remain at 10% of the city budget. But the more reasonable assumption is that payments on infrastructure will increase to become a larger proportion of the city budget -- particularly when we're talking about, say, sewers, which won't generate additional demand for policing or schools (or at least won't double it). If his claims were accurate, no suburbs so defined could have been built in the first place. At the very least it's annoying that he doesn't show more of his work, which is easy to blame on the fact that he obviously wants to destroy our way of life.
Uh huh... "our way of life" that really only came into existence after WWII. Strong Towns focuses a lot on how cities used to be in the US, and points to that as a pretty good model. They grew incrementally, both up, in, and out. They mostly grew thanks to local, smaller efforts rather than mega projects.
The focus is more on suburbs, which are more of a recent invention, especially the US-style ones where no one can live without a car and all the subsidies that accompany them.
Realistically, not everyone is going to live in some dense urban core, and there'd be a gradation from that to rural, like there always has been. The point is that should probably be a bit more natural curve, rather than having tons of burbs that don't really pay for themselves, long term.
But in any event: a few hard-core libertarians would like to completely throw out zoning, but most people just want to bring it back in line with things that are actual nuisances - or bring back actual nuisance laws in some cases.
We've gone from legit things like trying to keep factories away from houses and schools to people losing their marbles about a duplex being built in their neighborhood. It's gone way too far.
As to why markets are generally a good way of solving complex problems with lots of moving parts, I did mention that in the article.
I don't think there is any limited supply for suburban neighborhoods.
But there is a supply problem for dense urban living with lots of services within a short walk, and public transport to work. Demand is higher than supply, which has made prices high.