I have no idea, but I suspect it is combination of factors.
Let me ask, does Spain have and enforce loitering laws that prevent homeless of camping in front of businesses? Is this a factor in driving them to under the bridges instead of roaming the streets?
Because CA prohibits the police from enforcing any of these laws, and many prosecutors refuse to charge anyone with "low level" crime.
SF and LA (and CA in general) has the nicest weather, unlike other parts of the US where you have Extreme cold, extreme heat, or both in many area's LA and SF have mild temperate weather with limited rain fall, this promotes a "roaming the street" style of living.