If you explicitly went up to an Ivy League or top liberal arts student with the above quote, most of them would be disgusted. It's not like people walk around campus all day talking about how everyone else is inferior to them. Someone like that would probably be considered an antisocial boor.
But the point about not having anything to talk about with people who didn't go to an Ivy or other top college is true. One of the selling points of Ivies & top liberal arts colleges is that they "teach you how to think"; well, if they teach you how to think, how do you relate to other people that think differently? There's a shared depth of experience that going to a top college gives you, like the close relationships with professors, chance to talk to well-known public speakers, hanging out on a lush green quad in the middle of a city, or being able to break lots of rules without the administration caring. People who went to state schools, community colleges, or no college at all often didn't have that experience, so what's there to talk about?
This is ameliorated somewhat depending on what you choose to do after college. I went into computers, where many of the best programmers went to no-name colleges or dropped out of high school, so a.) I can always talk tech with other people in the computer field and b.) I found out pretty soon that where you went to school isn't a good proxy for ability. Some of my friends are bakers or teachers or secretaries, and they're sitting firmly in the real world. But the financial system and top law schools are populated with huge proportions of alums from top colleges, so the "Amherst bubble" (which includes Harvard & Yale & Columbia & Brown & Dartmouth & Swarthmore & Williams too, except they undoubtedly call it by their own school name) extends over it too.