Conclusion: the people selected to walk on the moon were fit and healthy
(Also yes obviously the sample size is too low to draw meaningful conclusions)
More likely though, as you suggest, the same astronomical standards (pun intended) applied to all crew members.
They could spacewalk and open CM hatch from the outside. Wrench for opening the hatch was stored in the lunar module.
It's not a stretch to think that the people who flew on the earlier missions could have been older on average, though. Just looking at some of the ages, 2 of the still alive crowd are younger than all of the Apollo 8 astronauts. All of them are younger than two out of the three Apollo 8 astronauts.
Even a few years difference in age can make a huge impact when we're talking about people in their 90's.