While I agree he was being more than a little dismissive, I think this is an important thing to understand: "Poor" doesn't mean the same thing to everyone.
When you grow up in the suburbs and your friends get brand new clothes every month when they get taken shopping, and you're lucky to get 2 new sets of clothes at the start of the school year, you feel poor. And the kid who grew up in a homeless shelter thinks you both sound rich.
Poor does different things to different people because poor is always relative. I was "poor" in NJ working in restaurants in my 20's and I never cooked at home because I worked 12-14 hour days. I still paid my rent, put gas in my car, and had money to buy drugs after it all. If I told someone who couldn't feed their kids what I was doing, they'd have looked at me like I was the biggest wastrel in the world. But when all I could see was the rich people rolling into the dining room day after day, I couldn't help but feel like I was at the very bottom of everything.
What you went through sounds terrible. And saying that it's just because you're lazy or stupid or "weren't being poor right" doesn't fix anything but just acts as a way to explain someone else's feelings/opinions. I wish you luck fellow human, because none of this is easy.