Property values is just one part of the problem. A lot of the xenophobia is also due these migrants bringing their political opinions (and votes) with them.
I know a guy who moves from CA to Idaho.
He complains people don't maintain their houses nicely enough.
He complains the dirt roads aren't graded well enough.
He complains that the super market doesn't have what he wants.
He complains that the way they do excavation is wrong (he's a septic guy by trade).
At literally every turn he complains that the locals are doing it wrong.
I've told him that the problem probably isn't the locals. (And I concur with what the other guy said about rising property values destroying communities.)