Consider the analogy to multiply resistant to antibiotics bacteria. They’re far more common in hospitals than elsewhere because hospitals are swimming in antibiotics. Once they leave the hospital environment they’re paying a cost to be antibiotic resistant with far lower benefits so they either lose the capacity or are outcompeted by other organisms in the same niche.
The explosion into a new niche does happen occasionally, when a new capacity is extremely useful or when an adaptation is discovered that is far more efficient but has similar costs or better than the alternative. I’m not familiar with any examples in microbiology but invasive species exist and so do we so we have existence proofs in macroorganisms.
So there is very little to worry about from alien world infections.
And also, incidentally, why most awful diseases come from Africa, and very few from the Americas.