> Are we designing low income housing wrong?
I'd argue yes. We should be building housing for working class / middle-class residents (sometimes called "missing-middle" housing, although developers often abuse that term to include luxury units) That's where demand is greatest while still having reasonable budgets and modest-but-workable profit margins for developers involved.
> Is anything getting in the way?
Yes, developers. (Occasionally zoning/historical associations/local laws in places like San Francisco or NYC. But primarily, it's developers)
Instead of building where demand is greatest, developers build only where profit is greatest -- almost every developer focuses solely on ultra-luxury housing (with ultra high profit margins) or low-income housing (where margins are reasonably good due to government subsidy).
If there was new working-class housing construction, their old units would filter down to low income households, because there's enough middle-class people for that impact to be felt in a market. But the margins there are lowest, so no company chooses to do that.
Instead, property developers focus soly on high end. Technically, those filter down eventually too, but by definition, there will never be enough high-income people for those properties to filter down to middle class folks in amount meaningful enough to help the market (so in turn, their properties can't filter down to low income households)
This breaks the entire market. High-end construction costs soars (because they want it and can afford anything). Middle-class properties become psuedo-high-end (for no other reason than lack of middle-class product) which creates this weird distinction between rich middle-class people who bought before the bubble, and poor middle-class people who have healthy wages, but burn 50+% of their income on rents (and so are effectively poor, even though they are on-paper upper-middle-class), and now you have an ever-growing group of "low-income" people who can't afford anything.
If we prioritized middle-class working-class housing, we could get units going without major government subsidy, and the old housing could become affordable to low-income folks (also without subsidy). But that would require companies to not prioritize profit over everything else, so it won't happen.