American worker doesn't mean "white Murican worker". It means American citizen. American citizen isn't a synonym for Caucasian.
You're doing that thing that people do when we talk about H1-B. You're conflating opposition to H1-B with racism and xenophobia.
It's not racist or xenophobia to say that American workers should have first crack at economic opportunities in their own country. As noted in my original remarks, this is something we have to talk about if we're serious about addressing things like income inequality and equal access to economic opportunity.
I think it's utterly wrong to give the best economic opportunities to foreign nationals when there are imminently qualified US workers.