I'm not sure why this is presented as fact. Most single-family homes in the US are humble midcentury boxes, or bland suburban copy/pastes, or hideously garish McMansions. None of these are any prettier to look at than a 5-over-1.
If maintaining an “aesthetic” costs keeping people in bad conditions, it’s not worth it.
And I agree, it’s a false dichotomy anyway as the most attractive development targets in a given area are the less desirable (ie less expensive) plots.
However, we can build dense housing that isn't awful to look at for only marginal price increases in many cases, and where possible, I think it makes sense to do so. No individual should have veto power, but I think it makes sense for a community to have a stake in building beautiful urban areas.