Caring about hours worked is for bad managers, plain and simple. The biggest reason to not work in the office is that it's easier to avoid these bottom-of-the-barrel, butts-in-seats leaders. With them out of the way, you're free to tune your output in such a way that keeps your employer happy while also freeing up your life.
If you're under-loaded, then a good manager can shift the balance around the team. Maybe someone else is over-loaded. If the whole team is under-loaded, then maybe the team can deliver more features. Maybe its a good time to pay down some of last year's technical debt. Maybe its a good time to take a bet on a higher-risk research project.
My peak throughput isn't necessarily there for the company's taking. I guard it as my greatest secret. My manager only gets 2 signals from me regarding load: It's good..or I'm overloaded. Never the truth when I'm underloaded if everyone's happy - that would just be harming myself for some company's benefit.