In my view, this is perfectly natural. Your emotions are tied to your expectations. As you grow older, you have seen more things, and better know what to expect. The first time you drop your ice cream cone on the ground as a child, you learn that a delicious treat can be destroyed so easily! When you drop your ice cream cone on the ground as an adult, it's like yeah, well, that happens sometimes, and hey, I've had ice cream a hundred times and I'll have it another hundred times.
In other words, since the second time something happens to you is less noteworthy than the first time, fewer noteworthy things will happen to you per year as you get older. Less noteworthy events means less excitement and a faster apparent passage of time.