The problem is there's no solidarity with the newcomers, and a strong desire by incumbents to keep the place the same and not turn it into skyscraper-filled Guangzhou. For a lot of people not building housing is an important priority in their life that they will fight for. Not to mention the question of money.
You might equally ask why the tech industry feels the need to be stuck so tightly to SF when it's far more amenable to remote work than most industries.
There's also the little matter of the earthquake zone.