It doesn't really have to compound. If you travel a few weeks every other year in your 20s or 30s it adds up to a lot more experiences than if you hadn't. When you get up to 40 it makes a meaningful difference.
Before the pandemic you could take a flight almost anywhere for less than $1000. Not using that opportunity ends up defining people, or at least their situation, whether they like it or not.
And don't get me wrong. I understand that it isn't always so simple. Still if I had to choose I would choose the version of myself that is willing and capable of traveling.
Could you go traveling when you are 40? Sure, but it isn't necessarily the same thing.