Sounds like a no true scotsman: "if you didn't feel poor, you're not poor enough to count."
I think we could consider poor as having all of your needs but none of your wants. And if you have all of your needs and all of your wants, then you are absolutely not poor. I agree with you. You cannot 'be poor but not feel it'.
While it is true that children may not realize they are living in poverty, when they grow up, they look back and then realize the conditions. I can see the point of someone saying they never realized they were poor growing up, but if they look back, they must realize that they felt it but did not realize it at the time.