If you are a leader, and not just a manager, you will help your programmers improve their skill, so over time you spend less and less time managing them. Your programmers will appreciate it because with greater self-management comes greater happiness and job satisfaction.
The extra layer of indirection helps to account for uncertainties in the task, imprecision in the estimate, and chaos (in the scientific sense) in how long individual tasks take relative to aggregated historical metrics.
The drawback to units of effort is that estimates eventually bubble up to non-technical people who don't discern the difference between effort and time, and for most projects there are going to be some time-based constraints on the schedule.
You can still take a probabilistic approach to tasks that might interrupt you.