Of course, as you point out, to work like that sustainably it is important to pace yourself, limiting working time. The natural consequence of that is a need for prioritizing the todo list, because not all of it will get done. I say “the” todo list, because if there are multiple they must be merged so it can be determined what tasks come first. It is cumbersome and distracting to do this merging and develop a business understanding of everything landing in an inbox to be able to prioritize it so indeed a natural temptation is to let a manager do this toil, and just follow their cue.
The consequence of letting a manager setting priorities is a loss of control. You have to have faith the manager will give the right kind of work in the right amount. I am not a good fit for this approach, so I tend to take control of my todo list by proposing my own projects or seizing opportunities as they pass by. This does require having good relations up and down the org chart and good insights into the nature of the business, so ymmv.
Seeing work as finite is imho mistaken. The possible work that can be done in a business is effectively infinite, and an effective business will find the best set of work to do with the resources they have, and an optimal level of resources to do enough work to deliver value, but without breaking the bank.