This is an answerable question: the median American household allocates 2-3% less of its household income to clothing in 2025 than it did in the 1980s. That's about $2000, for the median household.
Also, the financial environment that created "cheap clothes" is also the environment that suppressed middle class wages, and drove the cost of housing up ridiculously. The two are linked. Pretending they aren't is just fantasizing.
Finally, while people are financing larger new houses, the old small houses and apartments are still ripping upward in price. It isn't like people can just choose to pay 1980's prices for 1980's housing stock. They are stuck paying much inflated 2025 pricing for 1980's housing stock.