I've been supporting an advocacy group looking to build social housing and at least half the pushback is from anti-housing "tenant advocates" that work for non-profits funded by extremely foundations with boards that are all very wealthy with multiple homes, and don't see the need for more housing. The "tenant advocates" seem to view housing similarly and only support public housing to the extent that it doesn't actually get built.
Unless you are only talking about housing for a certain type of people, and that you'll only approve housing for that type of people, otherwise no housing? That's a very common position that I disagree with strongly.
Soak the rich, let them pay for the high cost of new housing, and it benefits everyone.
For public housing, it should be for all incomes so that those with high incomes subsidize those with lower incomes, so that the public housing has support throughout all levels of society when it comes to vote for maintenance every year, so that when there's a budget cut it's all of society fighting to ensure that the public housing is maintained.
We need to break out of the idea that only some people deserve to get housing, and that we should build just for that narrow band of people. New housing is expensive, we need a lot of it, and those with money are the ones who should pay for it.
A subtler reason is that public housing is drastically more expensive to build than private housing. There's multifarious reasons for this, and the underlying problem is absolutely worth solving (it's one of the three big prescriptions in Abundance), but it's a bigger problem than housing. Right now, people are focused on whatever's going to bring the most units online fastest.
Bringing more units online won't matter much if people can't afford them. Zoning (which is what I take abundance to refer to) can only do so much—we will need to attack housing as a commodity (or an asset, depending on who you listen to) in order to make a dent on the housing issues outside the upper middle class.