Depends how it's done I guess. France has a decent population tower, for lack of a better expression:

https://www.populationpyramid.net/france/2026/

If they keep it up, their population will remain stable up to the 22nd century. There will be no further noticeable ageing and thus no threat to anyone's retirement.

Decline fails to uphold the prevailing world economic ideology. That's the real problem with population decline the way I see it. We will need to adjust in ways that don't support the idea of "unlimited growth".