It's not so simple. Wealth can be used to gain access to limited resources such as land. Greater wealth inequality means a small segment of the population can afford to buy up all of the housing, driving up prices and blocking others (who may not otherwise be any worse off than they ever were) from owning land. They can then perpetually rent out this land, creating an effortless income for themselves and siphoning value from everyone else.