What we're fundamentally speaking about is the
scale of things. I'm certainly not aiming for a level where immigrants have no impact - I think controlled immigration where immigrants assimilate, and also bring their own flavor to society, is a great thing and win-win thing for everybody. But there's a point where if you bring in too many people you start essentially wrecking the host culture and causing social instability, exactly as Western colonization did to the various indigenous cultures in place we colonized.
Based on US history, the number seems to be around 15% of the population being foreign born at any given time. This ballpark also seems to be playing out throughout Europe as well. Germany's at 20.9%, Sweden ~20%, UK 19%, and so on. The exceptions that exist like in the Mideast are quite different. For instance Saudi Arabia has around 40% of their population foreign born, but those are all just temporary workers who will never be granted citizenship, and live under a very two-tier system. The temporary residents do not have anywhere near the social impact people who might eventually become citizens, or live in the country forever, do.
Now getting back to fertility collapse. A shared fertility rate of 1 will trend towards you losing approximately 2.5% of your population per year, compounding. So after 80 years, a single human lifetime, you'd have lost 87% of your population. If we're speaking of maintaining population levels with immigration, you'd need to have 87% foreign born at that point in time. So what I'm getting at with the 15% is we can argue whether it's 10%, 15%, 25% or whatever else. But whatever the exact number of 'good' immigration may be, it's much less than 87%.
And furthermore, when you start looking at the actual numbers involved here, it is literally impossible. If we say e.g. 60% of the world, by population, is losing 80% of their population of that timeframe, then there literally are not enough people to replace them in the rest of the world. That'd be a population decline of 48% (of the global population) and only 40% of people in the rest of the world. And that's if you're willing to just accept literally anybody, regardless of whether they have any skills, education, or can even speak your language. It's just not realistic at all.