90% of people will not get past laying a concrete foundation.
Mainly because it's a good idea to leave certain parts to the pros. Anyone can do framing or hang drywall. Roofs are as simple as you design them to be. It's the areas where your house 'interfaces' with the rest of the world where you'll need the most help, but a lot of those situations can be set up so you can do the majority of the labor (and the tools available now make it easier than ever).
The USA has enough cheap and cheerful (aka "shit") housing already. The last thing we need are people with no building skills building crappy stud framed square boxes. If they have to build, let it be more quirky and individual, and if they don't, lets try to keep pushing building standards forwards, not backwards towards the level that Jill & Joe Everyman will likely (not necessarily, just likely) operate at.
I could take a random sample of 10 HN users, ask them to frame and drywall a house, and the result would have to be condemned. Please don't kid yourself.
Like you said, it's not the hardest thing in the world, but it's not something that just anybody can do.