The "ultra-wealthy" are not where the money is. Besides, what many leftists actually end up saying by focusing on those is that Gates and Buffett should not be allowed to put their money towards eradicating malaria, giving every African safe drinking water, and countless such high-value causes. Instead that hard-won wealth should be taxed from them and frittered away to fund a one-time super binge of mindless political activism and foolish "green" boondoggles in the highly developed West. Needless to say, I think this is a very bad idea.
You want to tax Larry Ellison's yacht and other ludicrous spending, fine by me. But that's the ultra-wealthy's consumption - it's not even their income, much less their wealth. If you can't tell the difference, you're not getting the point of this debate.