It’s not just about being able to live on savings. When you come from a poorer community you can’t really afford to take on an expensive education and mediocre pay, because the outcome of that expensive education generally requires you also support your parents/family that helped you get there.
But moreover yeah, richer families can do more outdoorish activities like mountaineering and that feeds back into the loop of even being interested in something like this.