I've been struggling with this part. I just moved to Austin, looked at the market from 2012-2020, and what do you know almost every house I look at has doubled in value in 8 years. That's a 9% annual return! Shouldn't I get the biggest house I can afford since the city is growing so fast?
Well not so fast. I have to subtract from that about 3% for interest, 2.5% in property tax, 1% for maintenance. Not to mention HOAs, PMI, insurance, etc. I might end up making 2% annually compared to 7% in the stock market and that's only if I see the same returns as the most cherry picked period I could possibly find. Anything less and I'm absolutely losing.
I agree on the you have to live somewhere part, but treating it like an investment seems like a bad bet.