It’s so simple:

Real estate is investment and no one ever wants their investments to devalue. And the only way to slow stop or even decrease housing is to devalue it by building beyond what’s needed. No investor will ever do that.

It is capitalism in its purest form. And no kind of free market will fix this issue.