(1) Jobs. There used to be a ton of electronics and garment manufacturing jobs in the region. Most such jobs required no more than a high school education, possible some vocational training, but the R & D and white-collar sections also provided employement for the college-educated. That's mostly all gone to China, Mexico, Indonesia, etc. Right out of high school I actually did a couple years doing electronics assembly line manufacturing in various locations, including in an exciting laser-welding shop, making ridiculously good money relative to my peers (I could afford my own apartment at 18) - and there were people who'd been there decades. Gets boring fast, so I went to college, as I didn't want to be there forever.
When you look at the huge homeless population today, I think it was less than 10% back then of what it is now, and my guess is that the availability of those jobs played a big role in that.
(2) Housing. The reality is, either you are in or near the 1% by wealth or you can't buy a home, and rents are ridiculously high, and there are confounding factors like AirBNB taking homes off the market and foreign investors using real estate as a tax haven / money laundering scheme, etc. Developers don't want to build high-density affordable housing as profit margins are lower. Increased population puts strains on public transportation and traffic. The whole region could turn into something like Mexico City (wealthy center protected by police, surrounded by miles of slums and shantytowns). Importantly, people who serve key roles - teachers, service workers, etc. - generally have to commute into the city, was they can't afford local rents. This wealth/poverty class divide is not very stable or pleasant to be around.
I think what a certain sector of wealthy SF wants (basically the people behind the recall effort) is for the police to mainly serve the role of kicking the homeless out of the pricey neighborhoods and concentrating them in areas where they never go, which will heavily impact these poorer neighborhoods, exacerbating the growing class conflict. Oh, and one job sector that is growing? Armed private security. I don't think the outcome of the recall will affect any of this significantly, however.
The only real fixes I see would be a massive jobs program to bring back manufacturing to the region and a push towards Tokyo-style tiny apartment construction across the region.