I don't disagree with you. The average eight hour office worker probably only does do four hours of work a day. But I think the problem is that that nearly everyone has internalized that cadence.

If every office worker had their schedule cut to a 20 hours a week, I strongly suspect that only 10 hours of work would actually get done. It's just too much of the low efficiency is culturally ingrained on every level from the cultural to the organizational. It would probably take a generation for things to change.