When you're older, you're more able to process concurrency and your sense of self is more compartmentalized. You have a work-line consisting of your career and job life, a family-line, a vacation-line that emerges when one travels and is otherwise ignored, and probably several friends-lines (college friends, work friends, friends of the families) corresponding to various distinct social groups. Each plot-line is allocated only a fraction of the time given to the single plot-line a young child has, and thus each progresses much slower per year.
For a child, a year is an eternity. For an adult, it can be very short, because many plot-lines advance very little in this time. If you have a vacation spot you visit every year, every time you go you are bringing up year-old memories that seem as if they occurred yesterday, so time seems to be going extremely fast.