So lack of supply isn't the problem, it's the people who want a place to live who are the problem? Would you make this same argument if the increased demand was from people being born rather than immigration (because it is both)?
I think if you want to be able to block your neighbors from building denser housing on their property then that should only be enforceable if it was agreed to by everybody in advance when they bought the place (e.g. via an HOA contract or some similar type of restriction on the property deed). Otherwise this is tantamount to allowing Walmart to get a vote on whether to allow Target to set up next door.