As you suggest, the exact calculus is difficult, but you also can't ignore the positive effect that many (not all) billionaires have in the process of generating their wealth. Amazon, for example, saves consumers' time, and makes it easier to compare goods and prices, saving consumers' money. Bezos became a billionaire because he provided billions of dollars in value to regular people.
That does not absolve Bezos or other billionaires of their sins. But likewise, you cannot exclude it from your calculus.