The vast majority of the extant housing in the US was built for a world with cheap energy and air conditioning. Take away the AC and it needs extensive refitting to even be kinda habitable.
[EDIT] OK, that's overstating it in the North. Newer houses & apartments sans AC up there would just mean a couple dozen nights per year sleeping outside, and having to be way more diligent about fighting mold and mildew. Tolerable, but not great. The South, though....