You clearly don't have these issues, and I doubt you care about anyone who does, because your "Oh, come on." response positively reeks of 'that's not my experience, so other people's can't be that different.'

It's never just one thing like travel time, it's scores of steps in a routine, which aren't always the same, and can easily be derailed by anything unexpected. You can estimate how long something you do frequently will take, based on how much time it took previously, and still get an inapplicable answer because distraction is a constant problem, and the executive function deficit means you literally do not think 'don't do that, get back on task' in the moment. You know how long everything should take, and still struggle to apply that when you're doing the routine.