I notice it very much when walking in my day to day life, although it also happens in car rides. As some others have expressed, it feels to me like my brain is more active and focused on the destination, whereas I'm more likely tired and docile on my way back, slipping into the rhythm of the trip. Specifically, I'm thinking of this in the context of a night out (alcohol or not), where incredibly long walks are feasible on the way back. Obviously, there are other psychological factors that influence perception at this point, but I notice the exact same effects all the time, just slightly less pronounced.

I also wonder about what I'll call the "Lifetime" effect. Consider how car rides in general or long wait times at a doctors office start to feel shorter as you grow up (infinitely long as a child, pleasantly short as an adult). Intuitively, this is because of your life experience (or lack there of) and the fraction of your life-to-date that any given wait takes up. The time lengths stay relatively constant, while they progressively eat up a shrinking percentage of your life-to-date. Perhaps this same effect could happen over the course of a day, or a couple weeks, or an hour. The initial trip consists of 100% of your experience thus far until you get there and subsequently turn around, but at that point any task necessarily takes a smaller proportion of the overall time than the initial trip felt like (and was) at the time.

Perhaps something like that would explain it as well.