I think many people do not fully appreciate the prejudice faced by people who were born into poverty. Growing up poor marks you with affectations of the underclass that you do not even notice but which everyone else does. These tend to cause people to discount you regardless of your intelligence or work ethic, even if you are no longer poor, because they can tell you were "poor white trash" or similar. Even if you are financially middle-class, you are still treated as low-status individuals across broad parts of society at large in many respects. And they know this. If you look at it from the perspective of someone in poverty, the effort to change your financial position does not come with a concomitant change in social status and respect.
One of the most consistently successful strategies I've seen for people from the underclass to greatly expand their opportunities in life is to carefully and systematically erase any affectations, behaviors, and preferences that mark them as a member of the underclass. As a society we like to believe that intelligence, discipline, and hard work are enough for anyone to "make it" but the reality is that for people born into the underclass you often have to spend a lot of time finding ways around social prejudices as well.