What's striking is how hard it is to take what's already present and build up, to modernize or expand existing infrastructure. Construction in a city is expensive, and a great many existing buildings break various codes (ADA, asbestos, etc) in ways that are more expensive to fix than from-scratch development is.
When building new is cheaper than retrofitting, people will tend to do it. Putting up a new building in Boston's financial district would likely mean knocking half a dozen valuable apartment buildings, and won't ever happen. Putting that office building up five miles away is cheaper, and Boston is already seeing a number of industrial parks on its outskirts for that reason.
The exceptions are places where land costs a fortune (NYC), zoning is impossible to deal with (San Francisco), or the sprawl is already so atrocious that "outskirts" is a death sentence for a new venture (LA).
Subsidies for expansion certainly matter, but even without them there are substantial incentives to choose sprawl over infill and development.