Wealth follows the wealthy, and most of the wealth in the US rests in very few hands, much of it offshore and in other ways hidden from taxation. If they leave, the "wealth" leaves with them for the most part.
For another nation to get to the level of US it would need all of these things and it would take decades and massive investment to build up all of this. Few very rich people taking their money and leaving for some other country would not be the end of the world.
All true enough, but I'm not convinced the wealthy would leave the US en masse as it gradually declines in power and we enter a more multi-polar world, with the dollar possibly losing reserve status at some point. Where would they go, China? I think in the scenario where the US is less powerful compared to the rest of the world, it's still a very attractive place to live in many ways (edit: especially for Americans).