Anecdote: When I was younger, I used to take a public bus to a local bookstore and work through the SAT practice books (on separate scrap paper of course).
I'm not saying the poor aren't disadvantaged. It's just that the disadvantages are more along the lines of attitudes towards education in the first place (along with everything else). There are also fewer people nearby that serve as an "example" on how to learn or get ahead.