If you love what you do, you'll never work an hour in your life. So maybe academics shouldn't work at all, since the compensation isn't worth doing something you don't love ;-)
Also, the implicit assumption is that, if you love what you do, you don't love doing anything else. I want time to devote to activities other than the work I so love. Much of that time can and does end up producing useful cross-pollination with my day job, but that's just gravy.
Swinging a blunt axe at a tree does nothing but keep you from devoting time to sharpening the axe.
I can imagine this is the same for academics.
Plus a lot of the job are things that they hate doing - politics, fundraising, dealing with hopeless students, writing modules that never get used. Many of them spend all their time doing these things, and the "overtime" is necessary to get the real work done after all that.
I am currently a postdoc. It's a scam that academia perpetuates this idea to justify paying less.