Just as any bird has the "right" to go wherever it wants and any other animal hat the right to go wherever it wants, I have the natural right to go wherever I want.
I'm a free human. As I'm born on this planet, it's my right to go wherever I want to go on this planet.
That governments want to take away that right of mine, that liberty, is just one example of how governments are using the tyranny of power to force people to their will.
That governments are just the thugs with the best weapons is just Realpolitik. This still doesn't invalidate my right to freely cross borders. I never signed away these rights.
Though I might be stupid to try to exercise it.