Colleges spend a lot of time and money on selecting the smartest students, so companies don't have to. Screening is quite expensive, and anecdotally, people from prestigious universities generally pass interviews at a much higher rate.
Being around smarter and more ambitious people makes you smarter as well. I feel that has a lot more impact than faculty. My top CS school had some senile tenured professors who were barely comprehensible, so we had to learn a lot from each other.