You're making too many assumptions from such broad data. No, houses don't generally fall apart within 50 years. But most houses built 50+ years ago are small by modern standards, and many have been torn down and redeveloped since then. For the ultimate peek into this phenomenon look at Tokyo, where the lifespan of a house is short and any given plot of land is rebuilt several times per century.