You are not wrong. However, houses could still be built from wood like the 500 year old houses and churches made with actual timbers. Timber framed houses would use a lot less wood in the long run due to less waste and greater longevity (especially when coupled with bricks and concrete).
Milling large trees down to 2x4s and coupling it to drywall to build homes that don't last is a travesty.