Where parts of San Francisco are jammed with houses, every nook occupied, not unlike parts of Manhattan or Boston, there are large swaths of the Valley with nothing but parking lots.
San Francisco is not unique in this regard. Every urban area in America is like this, including Boston and NYC.
San Francisco has a number very tall buildings with a few even built on the side of a literal mountain, and all of these have survived several significant quakes, so I doubt this is the actual reason.
The buildings on the hills are the ones that survive the quake. The buildings built on sand, like in the Marina or Mission Bay, are the ones that are vulnerable to earthquakes.