Hiring is actually a tremendous burden for most small/mid sized business with all of the compliance and regulatory stuff. Its far cheaper to hire one person for 40 hours/week than 2 people for 20, even if their pay is half.
Is it actually true to say there is an market equilibrium at 40 hours? I'm no expert, but 40h seems heavily reinforced by regulation (FLSA comes up on search). Without government intervention, I'd not see much stopping a race to the bottom, at least for a significant part of the job market.
I wonder if less wealth inequality would give more leverage to the average citizen, enough so that this additional bargaining power would make a significant difference on the job market. Influences like that might explain the systems trends over longer time periods...