It depends on what you're restricting and why. Restricting access to things based on age can absolutely be ageism if the thing does not need to be restricted.
The burden is still to demonstrate that a restriction is wrong. If that can’t be demonstrated, then labeling it ageism is a purely emotional appeal.