Unequal gains is the problem, and your intuition can help inform why: the 2023 dollar index used is based on what rate of inflation? CPI? RPI? Something else? Why is it that in 1970 a median income household could buy a home on a single income and raise a family (including sending kids to college), on a 2023-dollar income of $66k, but that's mostly not possible on $106k in actual 2023 for most households.
When you adjust for real buying power using a less favourable means of assessing inflation and taking into account housing costs more fully, I sense you'll find that the bottom two brackets are behind their 1970-adjusted counterparts, and the upper income bracket is significantly better off, especially if you then break that upper segment up a little into more categories.
And, without something happening to adjust this, the effect is going to just get worse and worse, and everyone knows it.