Wealth isn't something that be quantified objectively, so at some point you need to put hard parameters on what qualifies wealth or thresholds of wealth. At some level, "money" (which is the representation of wealth) is at any given point finite. This makes it inherently a zero sum game.
The "any given point" allows us to avoid arguments about fiat and monetary policy. At any given point if I get $1, that value comes from someone else (or elses) down the chain.