Exactly. I've often felt that a lot of the "wow humans are so irrationally risk-averse when it comes to money" conclusions that some researchers make are suffering from limitations of extremely limited models.
For example, imagine a stranger comes up to you on the street and offers you a coin game, where if you win you gain $1 billion, but if you lose you owe him $1 million. The simplified model would claim that you should play the game because the expected value is positive. A more nuanced model would understand that being that much in debt would be a lot more suffering for me than the offset from whatever I'd do with a billion dollars. Even if the coin was weighted more in my favor, and the penalty was less, an even more nuanced model would recognize that I'm probably doing OK enough in my life with the money I have and that anything to chip away at that is an unnecessary risk.
And then a model that even more completely captures the interplay of reasoning that humans do would acknowledge "wait a second, who the hell is this random stranger offering me a devil's bargain?" The researcher with a simple model would say "oh, just ignore that, just take the situation at face value," but that's never how any decision is made. Our priors for "suspicious people offering bargains too good to be true are actually trying to cheat you somehow even if you don't know how" is very high. This is embedded in cultural/religious parables that teach "you do not make deals with clever demons," and because the culture that taught that has survived and persisted over time, it's probably a good prior to hold onto.
All that complexity is lost if you only think in simple terms of expected value and single-iteration games.