What matters even more though is the student body. Hanging out with a bunch of really smart and dedicated young people for 4+ years gives one a huge leg up intellectually.
At my small school there were very few smart people to talk to, it was very lonely and insular. By comparison, time I spent hanging out around Berkeley and PENN were very exciting intellectually.