I'm not convinced. Passports might be a modern invention (and really that isn't surprising, a global identification system requires sophisticated bureaucracy and cheap, accessible transportation), but that doesn't mean that people used to be able to move freely across borders. Modern technology makes immigration accessible to the masses in a way that simply wasn't the case in the pre-modern world.
A few things you would have to consider before immigrating in the pre-modern world:
- Where are you going? There's no internet, books are rare, and maps are inaccurate (and often plain wrong)
- How will you get there? Transportation is expensive and unreliable
- How will you preserve wealth? A global financial system accessible to normal folks does not exist yet
- What will you do to support yourself? Pre-modern communication infrastructure makes scoping out foreign labor markets difficult
- How will you learn the language? You'll have to find someone to teach it to you, instructional material will be limited
- Who will you leave behind? Any family members that don't come with you will likely never be seen or heard from again
- What if people decide you're not welcome? Well-enforced protections against discrimination do not exist yet
People did immigrate, so obviously immigration was not impossible. Not all of these questions apply to every situation (people migrating to an adjacent country would not have to deal with many of these concerns). But immigration was far harder than it is today, and fraught with risk. Pre-modern borders were, by virtue of being pre-modern, effectively closed.