> This means that investors will buy up even more
Not necessarily. Right now, 70+% of house sales are to owners who will live in them. If they're (mostly) gone, house prices will collapse, and some existing investors will get spooked and start selling, driving prices further into the ground. Many investors will vow to never buy another house, and others will stay away.