Hans spends an awful lot of time poking holes in the historical accuracy of income data. But Pinker states in very basic, understandable terms that the benchmark isn't the point — setting the benchmark at $7 or 100k doesn't stop the fact that wealthy has trended positively. Starting from 1981 or 1810 doesn't matter because it's relative: Do people have access to more wealth (things they desire) than before? The answer is yes. The other benchmarks of education and literacy round out the picture — wealth and money isn't the only thing being measured.
As much as Hans paints a utopia of open fields and shared communal livestock we know people historically chose to pillage each other's resources. Without incentive to look beyond differences we went to war and took from each other's cultures to accumulate wealth.