SF is a highly compartmentalized city. If you walk through some areas to go to work, you see it. If you live just 400m away, you may never. That actually may possibly be part of the problem.
I was in Berkeley for a couple years, living in the hills and working downtown. The two areas are just a 20 minute walk apart, but you wouldn't ever see the deep social issues and poverty on the streets if you never left the hills.