Real estate speculation by putting all your money in a your own house is generally not a good investment. You aren't diversifying, you don't have any information advantage over the market, you're unlikely to be objective about the value, and there are high friction costs in buying/selling it.

Maybe a bit like driving an classic car in everyday use. Except the car has low friction.

Edit: Not saying you should not buy a house, just that you should be honest about what it is - you get value from owning and living in it, and the market risk is unwanted for most people.