I grew up in poverty, on food stamps. I do somewhat well for myself. My wife had a similar childhood.
We have poor financial habits that occasionally hint (I won't go so far as to say that they "threaten") at upending our lifestyle. Just having "more money" doesn't protect. Our habits would, if not kept in check, squander nearly anything. We are, in many ways, our own worst enemies. I've struggled to recognize this my entire adult life.
I think when someone starts speaking as you just have, it has as much to do with social posturing as anything. You have to say that, to show everyone else how special and empathetic you are. You understand the plight of the poor, that it isn't fair and they're "just like anyone else". That if some windfall dropped into their laps, they too would be rich forever, because that's how wealth works. And when you do this, you see everyone clapping and murmuring to each other about how enlightened you are, and you get off on that.
There are better habits. The wealthy tend to have more of them (many more), and the poor never develop those at all. Mostly, I think, because if they have these habits for any length of time, it quickly becomes incorrect to refer to them as poor. These habits can't protect against all misfortunes, but they protect against some and they mitigate the rest. And when you teach people that these things aren't true, you're doing them a disservice. You're robbing them of the one thing that might change their lives. It's both bizarre and cruel.