Comparing the 2 densest boroughs of NYC to the entirety of the Bay Area seems pretty specious to me. Last time I looked, there was a giant sprawl of suburbs around that city too.
San Jose's 5,000 per square mile underestimates what it feels like to live there.
See the borders: https://maps.google.com/maps?q=San+Jose,+CA&hl=en&sl...
All that area in the southeast are farms and mountains. A The entire northern part are marshes. Its "effective" density is really much closer to 7-8,000 per square mile.
Los Angeles is also one where the density numbers underestimate the feeling of it. https://maps.google.com/maps?q=Los+Angeles,+CA&hl=en&...
A lot of that land is mountains. Indeed, south-central and central parts of LA have about the same population as SF and pretty much the same density: http://projects.latimes.com/mapping-la/neighborhoods/populat...