They just don't fit in.
I was the best student in my shitty high school and so I applied and happened to get into an Ivy League school. The environment there was so incredibly different I just didn't fit in. Super-preppy good looking kids (they all are thin and tend to look good) who are invariably very well-spoken, totally expectant of all the resources lavished upon them, and just know how to talk to professors and interview for jobs and so on.
I was of a different breed. Ugly, fat, not well-spoken, no idea how to talk to professors, no gutso to challenge a nobel prize winner on his opinion. I was also somewhat shocked with the amount of resources provided to students. For example, for unpaid summer internships my college just gave out $5000 in living expenses so undergrads could do any unpaid internship they wanted. That's about a quarter of what my family earns in a year. Not complaining, of course, but I hope you see how alien this environment was for me. Same was the case with interviews, I didn't have a suit and I had no idea how to bond with an interviewer over our shared love for some exotic sport like skiing.
I just didn't fit in. If I were to do it over again I would pick my Ivy League school again, but I recognize that it is an alien environment. I can totally see why another low-income student would be scared by all this and just not apply.
PS: If I went to a school like MIT maybe my experience would be different. I went to one of YPD, which are traditionally more preppy.