Investors that buy houses just rent them out again, so the houses are not removed from the supply of housing, just moved from the homeowner's market to the rental market. There is nothing wrong with this - renters need a place to live too! The problem is the lack of overall housing, not the balance between homes to rent and buy.

The fact that it's profitable to buy homes and rent them out means that renters are currently overpaying relative to buyers, and shifting properties over to the rental market will help correct this inequality. If you advocate against this then you are arguing that renters should be screwed over the subsidize homebuyers.

If you want to screw over wall street you should build more homes to flood supply and tank the value of their investment. Of course many of the people who complain about Wall Street buying up homes are also NIMBYs who adamantly block all new housing because they are supply-and-demand deniers or vacancy truthers.