It does cost a lot, but in most cases you're going to get more bang for your buck. (and you can also forgo features you don't like or need: like a homeowner's association; or pre-existing utility connections you don't want).

The other thing about building a home - particularly if you're in a high-demand area, the appreciation in the first 2 years is pretty insane.