As anyone who has worked in undergraduate admissions will tell you, it is far more complex than that. But when n grows prohibitively large, you have to go with solutions that are less than ideal.
My alma mater, thirty years ago, eschewed the traditional college application in favor of a more homebrew approach: since it was a small liberal arts college, it felt that the best way to see if a student was a fit for the college was to place him there for a span of a week or so: meeting with professors, faculty, taking a tour of the place, making sure things 'clicked'. He or she took an exam at the end of it, focused less on trying to define the student intellectually and more whether or not they were engaged with the college: there were quizzes on campus trivia (we're the second oldest college in the nation, and talk about our history incessantly), open-ended questions about what they liked, et cetera -- all designed to see if the student was really meant to be a part of the college.
This worked twofold: it let the admissions faculty see whether or not the student was really taking the experience seriously, and it let the student see whether or not they wanted to take it seriously.
(Why no math tests? When a professor spends a few hours with a prospective student, they can generally deduce whether or not they're smart enough to handle the rigor.)
Today, such a practice is impossible: the number of applicants has increased by at least 20x. There literally are not enough days in the calendar.