> Few people in a neighborhood of single family homes would want a 20-story apartment tower getting built next door.
Only this is just a thing that really doesn't happen much at all. Very few developers are putting up 20 story apartment buildings in the middle of an existing SFH community in the suburbs somewhere. Most of those such fights are people fighting the 5 over 1 development built greenfield just outside the current SFH neighborhood in question. And for those such people? They should have zero say.
If they are doing so, it's typically in very dense city cores that support the density and thus investment. If someone wants to put up a 20 story building next to me in my Chicago neighborhood, sounds good to me! I am upset there are not more of them and that zoning has effectively made them illegal to build here, as I'm roughly 500 feet away from a rapid transit station and two arterial roads with high frequency bus routes. It's a crime that they are illegal to build.
I'd be an asshole if I bought a house in such an area and wanted my little conclave to be protected at the expense of everyone else in the city. I should rightly be overruled and told to pound sand and move the hell out if I don't like it.
Instead, laws have enabled such people to effectively veto such buildings. Or even simply stuff like 12 unit 4 floor apartment buildings that already exist due to predating zoning on my block.
Many areas of the US are effectively retirement communities by zoning. People who moved in 20-30 years ago who demand zero change and have captured nearly all political power.