It doesn't cost that much. The land costs a lot, and permits can cost a lot. The labor isn't that much, and materials are cheaper than ever (though most houses in the past were built with free labor with material prepared on site and so you don't realize how much it costs as nobody was getting paid)