Your perils are legion when you're suddenly the target of a squad of ninjas, or surrounded by a hostile armed gang. There being a social stigma attached to being rich or there being higher expectations on you because you're rich are miles away from a legion of perils. "My friends look at me differently" or "I tend to take the easy road whens stuff gets hard" are drastically less severe of problems than "I can't afford my kid's medical bills" or "I'm having a hard time providing for my family while saving for retirement". Yeah, their lives have problems, but that inevitable - if our lives didn't have some risks or tradeoffs, we'd go insane from boredom. But their tradeoffs and risks are very far removed from their physiological and safety needs.