I think the argument can be made that for the same money someone could build a mansion that also doesn't look awful.
I don't think this logic adds up. People care what the house they buy looks like, and some people even care what their house's appearance projects as an expression of their own taste.
People don't close their eyes when the buy houses, but neither do they have an eye for architectural nuance.