For a few years, I've been challenging people when this discussion comes up to give examples of situations where increasing housing supply resulted in falling prices. As far as I can tell, without a fundamental demand drop, this rarely if ever happens.
And you've hit on one of the reasons: the investors who want in want into a market where real estate commands the prices it does.