The reason there isn't enough housing, in SF especially is government intervention. Zoning laws prevent housing supply meeting demand.
In NYC, rent control laws are a major factor in the artificial housing shortage. There's a consensus across economists of all stripes that rent control is a bad thing ("the most efficient technique presently known to destroy a city -- except for bombing"). See the studies cited here. [2]
[1] http://pandodaily.com/2012/12/01/san-francisco-can-become-a-...
Or maybe I'm confusing zoning regulations with other codes.