There's a point where packing and leaving is easier. And the richest the people, the easier this is.
Governments have to realize that, at a certain point, they are in competition with other countries. Multinational companies and rich people come to mind.
The rich can pay their way through most of the immigration burdens that prohibit the average joe from moving (yes, even to the US)
You're absolutely right about the ability of the rich to buy their way to the front of the immigration line, I just don't see it happening in significant numbers.
What about the flight of manufacturing from the US, c. 1980-2000?
The missing piece is that you don't always have to choose. If the low tax rate attracts businesses (or even just causes corporations to arrange to report all their profits in your jurisdiction) you may end up with quite a lot of revenue even at the lower rate.
The key to that game is convincing all the other countries not to play. If you have low taxes and no one else does, you win. If you and all the other countries have low taxes, all the countries lose and the corporations win.
The U.S. strategy is kind of interesting: The corporate taxes are nominally really quite high, but the largest corporations (i.e. the ones best able to leave or rearrange holdings) don't actually pay them.