You also have to provide the immigrants with whatever social spending to which locals are entitled. They cost money; if the immigrants pay less in taxes than they receive in social benefits, they are a net economic loss. And, as you may expect, a poor immigrant likely pays little or even no taxes if their income is low enough.

Also, lowering the price of labor in a particular industry makes other people in it be paid less, or fired. As a result, they buy less, etc, slowing down the entire economy, influencing maybe your own wages.