Comment by triceratops | Hacker News Reader

Comment by triceratops | Hacker News Reader

Parent Full thread

> I still contend strongly that it is wrong to distill a person into a collection of numbers

No one's saying a person is a collection of numbers.

> My point in telling this story is that the raw score in a standarized test can be helpful, but it does not tell a complete story. It is part of a broader assessment

And that's very noble and well-meaning. But it can also be used to discriminate and deny opportunity. Raw scores, on the other hand, are hard data. You either have them or you don't. Do factors outside of one's control go into them? Absolutely. Those factors are all-pervasive and will influence anything you might use to determine merit.

> Kaplan, one of the test prep companies. They had very good data about how much their test prep improved test scores.

Of course they would say that. Even assuming it's true, Kaplan test prep books are only around $50 on Amazon. You could also get them for free from a public or school library (or the high seas, if that's your thing).

As I understand it, Kaplan has apps and dashboards and more material if you're a subscriber. Is there data about the marginal score improvement using that versus just the books?

When you get right down to it, everyone's starting point is different. Reducing the number of variables improves opportunity IMO.

My continued assertion is that "hard data" can be misleading, wrong, and discriminatory. It requires judgement to understand what it tells us and whether there is error in the collection of the data itself.

As far as Kaplan, I don't have metrics to share, but what I saw shocked me at the time. It appeared as though they collected data as to how much their test prep improved scores and it was pretty dramatic. And, they were smart enough to distinguish between their test prep books and their in person tutoring. In person tutoring worked better and by a lot, but it all helped.

As far as tests, I've never contended that they shouldn't be done, but that these scores need to be part of an overall portfolio. I have in fact sat on admittance boards for graduate school and the two dominant factors other reviewers used were GPA and where they went to undergrad. Both of these were hard metrics and full of bias. I'll also contend that they weren't particularly effective in identifying students who successfully completed the program.

Something to consider, I do think test scores can help screen candidates. For example, if one is going to admit into a math program, then their quantitative score should probably be above a certain level. After that, I personally don't think it matters. In that way, one could just set minimum criteria to be accepted, which includes GPA, tests scores, research, jobs, whatever, and then randomly select from that group. That would certainly eliminate a huge amount bias, though not all, but most people hate this approach because it contradicts this belief that we live mostly in a meritocracy. They want rankings and they want to know who number 1 is. My continued argument is that it's impossible to distill a person down to a hard ranking in a reasonable matter, especially with test scores, and it's not necessary for college admission since one can select a group of applicants for acceptance without ordering them.

I appreciate your willingness to engage and your detailed responses. I understand where you're coming from, and that you've seen how the sausage is made.

> In person tutoring worked better and by a lot, but it all helped.

Are money and other resources only helpful for getting a good test score? They don't matter at all for the factors that improve one's chances of admission? For example, going to a good high school district, participation in certain sports, hobbies, or access to internship, volunteer, or research opportunities?

My argument is simply: by making it about the test, you remove all the other factors where money could potentially have an influence. The rich students will get tutoring no matter what. But they can't gain any other advantages over the poor students.

> In that way, one could just set minimum criteria to be accepted, which includes GPA, tests scores, research, jobs, whatever, and then randomly select from that group.

That wouldn't be a terrible approach either. You are correct it would be unsatisfying.

> GPA and where they went to undergrad...were [both] hard metrics and full of bias.

I would argue they weren't "hard" at all. Undergrad is already gated by "squishy" criteria. Normalizing GPAs across universities (and countries, in the case of international applicants) is tricky and imprecise.

Money absolutely has an impact across easily available metrics. For example, money lets one purchase better coaching for sports or join a travel team. As we see from the lawsuit, athletes were accepted at a much higher rate than the rest of the applicants. Money lets one attend better schools either from a transfer or private school. Money allows for better tutoring. I completely agree.

Where I think we differ is that my contention is that money also allows one to game a higher test score on something like the SAT and by a significant factor. Yes, it would focus attention on this one factor, but I believe it also misses the point. College admissions is not meant to be a singular contest or game, but part of a broader ecosystem and that system has choices.

A university has a number of competing criteria and wants for its applicants. Generally speaking, a university does not want someone to drop out. They want graduates and part of screening applications is meant to determine whether they believe someone can successfully complete their degree. Some universities essentially operate like hedge funds that have classes. These universities have an interest in high net worth and connected applicants because these people can trade favors or give donations as alumnae. This is reflected in legacy admissions and automatic admission for certain classes of donors. Some universities want to give preference to in-state or in-city students. The purpose here is to ensure the university is locally connected because the state and city have the most influence on what is really a city inside of a city for most universities. Some universities want to give children of those who never went to college preference because it ultimately broadens their base and likely ensures their children will want to attend there as well. Some universities want to maximize their rating on the US News and World Report ranking because it means they can charge higher tuition.

None of those metrics above, moral or amoral alike, need a total ordering of applicants based on a test score.

As one other comment, when I sat on the graduate board, our department was flooded with applications from one particular country, more than half. Don't know why, but it was. Certainly, we accepted some of those applicants. However, consider the implications of accepting students purely from that one place. Would the local government be happy if we did not accept any local or domestic students? Would the federal? Probably not. However, if we accepted applications purely based on a singular test score, there would be incentive for that place to game the system and help with the test, legal or otherwise. No idea if they did then, but it wouldn't have mattered since we still read the broader application.

Reply on news.ycombinator.com