There seems to be very little distinction though when those high-skilled immigrants are being chased out of the country all the same, or start to avoid America because of that fear.
This nuance is important.
Perhaps for you the distinction is important, but in terms of policy, there is no difference at all.
If you start kicking out all the people who are brown or muslim or otherwise different, you hurt the society and make it harder for the legal high-skilled immigrants to live here.
(Not to mention that for three decades American politicians and corporations encouraged immigrants to come here and work without status. And that Irish and Italian immigrants came here 100 years ago before there was any immigration law; if today's republicans existed in 1900-1930, they would have demonized Irish and Italians for votes too.)