Almost no one in the US is calling for restrictions on immigration.
Tougher border control is restrictions on immigration. Why do you think various billionaire nutjobs want to build a great wall of mexico?
Moving towards an open borders would be changing restrictions on immigration.
But I'm guessing you're ok with the folks who migrated to North America. Just not ok when those same people move around their own continent.