There is of course, some effect but it's relatively minor for the middle class. Someone who's making a good income is not going to settle for the same conditions an illegal immigrant does, they're not competing in the same market. Your analysis leaves out a bunch of realities, for example: Most illegal immigrants share housing, reducing pressure compared to the normal. They also tend to stick to immigrant neighborhoods due to language barrier and need for cash transactions, this reduces their impact outside these areas. There is such a thing as homelessness more population on the lower income scale doesn't always mean that property values go up due to resources being taken since some will end up homeless. People tend to get the best they can afford, if illegal immigrants are taking the lower end of the market the legal people aren't just going to start buying "better" places.
Illegal immigrants have a negligible effect on the property values of the type of property people putting up walls and fences around their homes own. And lastly I highly doubt anyone who's against the border wall is realistically thinking "this will keep my property values up!"