The economy exists to support everyone's life, not the other way around. Just because you can argue someone can make a quick buck selling a basic necessity, that does not mean it's ok to deprive a whole community of basic necessities.
Take for example water. Would it be ok to hold everyone hostage over access to it, and even lobby for municipalities to stop the public sector from encroaching onto a service provided by a private business, just because Nestle and the likes bottle and sell it?
The current delta between the have and have-nots, and the profound impact that "small" real estate investments have on pushing access to housing far out of reach of your average middle and upper-middle class family, is a collosal social and economical red flag. Access to housing has a profound impact on everyone's lives and even community. You accept jobs based on access to housing, and the amount of time wasted supporting said job is time eaten away from the time you spend doing far more important things like caring for your children and family. Being deprived from that just because a faceless investor wanted to park his bailout money somewhere and decided to dump it in real estate hurts society.