It's totally unscientific but in my opinion it's like when you are hiking a path you never did and when you reach your destination, you hike back. Going back always feels faster even if the pace is exactly the same, but you already know the path, you remember some rock, or that view, or those oaks. And your brain works less.
We have routines when we are small children just like we have when we are adults so it might be that as we grow older the brain is in power saving mode and that feels like going back through a path you already know.