“Upper middle class” has nothing to do with it.
In 1950, the median household could buy a house with 2 years of income. That is, 50% of households could do this.
Today, the median house costs $400k. Only the top 12-14% of households make 200k per year.
So proportionally, the % of households that have the same buying power as a median 1950s household has shrunk roughly 4x.