When my ancestors came here in the late 19th and early 20th century, they were dirt-poor peasants in central and eastern Europe. I know at least one of my ancestors was a draft-dodger, too; he fled Russia because the tsar tried to have him conscripted to fight in the Russo-Japanese War.
The idea that the US always only accepted highly-skilled immigrants until very recently is false.