Taxing wealth is basically paying a rent to the society that allows that wealth to exist. Your land in manhattan is only worth $10m because society hasn't gone all Syrian. Your copyright holding over your film is only worth anything because we don't let people freely copy it.
When the wealth is owned by the people who aren't productive, you have a problem. It may be that those that are productive Leave and move elsewhere - who runs your hospital/golf course/power station then.
It may be that the productive say "enough is enough" and decide to invent the guileteen.
Either way when people have less to lose than you, it's a dangerous place to be.