I get your point: it seems like a moral good in America for people to have some inalienable (i.e. "not required to pay for") right to have some control over where they live. But it makes zero sense to have a black and white rule that everyone has a right to live anywhere they want. Do I have a right to take up residence inside your living room? Your home? Your porch? A public school's playground? The middle of an interstate? A public park? I think the reasonable answers to all of those are "no".
The US is geographically sorting itself by wealth. You used to have pricy neighborhoods but any given city generally had some amount of property available across a range of income levels. With the transition to a service economy, this is less true over time. Now entire cities are becoming uniformly expensive.
The reality is that I no longer think it's tenable to say that someone has a right to live in Manhattan regardless of their income level. The way things are going, it's probably going to be the case that the entire city limits of SF, Seattle, and NYC will go that way.
Does it suck? Yes. Can it be solved at the city level? I don't think so.