That is not true unless you consider having a mortgage equivalent to renting from a bank.
What is true is that in some metro-areas like Atlanta, 1/4 of sales (in certain price categories) are to investors.
I dream of a 100% tax rate on second homes, foreign ownership, and corporations owning single family homes and condos. Or even better, make it illegal. But any politician who pushes such a bill would immediately get removed from office and the law repealed.
Financial speculation happens because people believe (reasonably) that the price of a home will increase. Why do they hold this belief? Because the demand has increased while the supply is kept artificially constant.
That's what's so frustrating about all of this -- it's a complete own goal. If we all collectively do nothing at all, developers will move in to capture the consumer surplus. It will be profitable to build, not buy and hold.