In addition to the economic argument, I think there is a compelling moral argument to opening borders. Why should we deny equal rights (the right to work or move freely) to human beings based on their national origin?
If someone today told you the Jim Crow laws were economically justified in modern America - "we can't allow desegregation, blacks will compete with whites for jobs" - you'd see them as a disgusting, backwards racist. And yet in the US, foreign nationals are denied basic rights that US citizens take for granted. Does anyone believe these laws are actually just, or do we just support unjust laws we (mistakenly) think are to our economic advantage?