Having said that, his statement "If Harvard were really the best education, if it makes that much of a difference, why not franchise it so more people can attend" misses the point entirely. Sure, Harvard probably has above average education. However, the reason it's sought after is because of the perception that all your peers at Harvard will be extremely smart talented individuals. As a society, we're hoping that that by putting all these really smart kids together in the "Harvard environment", they'll go on create things that they wouldn't have been able to on their own.
The point I'm trying to make is that the exclusivity of Harvard isn't "an excuse to do something mean", but rather an effort to create conditions where our smartest youngsters can get together and really change the world. I do, of course, agree that there are many other ways of achieving this goal than just college education.