You can definitely build a whole house from scratch for less than 400k. You can build it for less than $100k - http://www.zillow.com/oh/home-values/
No, most of the value in a home comes from the land underneath it. The point being made above is that the availability of credit leads to asset inflation, as more people are able to compete for the same scarce things (desirable living locations).
Mind you, credit does have a purpose - if your income and your house's value appreciate faster than the interest rate being charged to you, then you were able to gather the resources for it.