Hacker News Reader

Comment by Dalrymple | Hacker News Reader

This is indeed a strange decision for MIT. The conventional wisdom has long been that the SAT subject tests are MORE predictive of future success at MIT, because the influence of test prep, cramming, test coaches, etc. is minimal for the subject tests. While there are reports of people raising their scores artificially on the non-subject tests by hundreds of points through these short-term methods, the subjects tests have long had a reputation as being more representative of what you really know.

View on HN

> The conventional wisdom has long been that the SAT subject tests are MORE predictive of future success at MIT, because the influence of test prep, cramming, test coaches, etc. is minimal for the subject tests.

Who cares what the conventional wisdom says? The psychometric results are that SAT I scores and SAT II scores predict performance about equally well in isolation and don't have more predictive value in combination than they do in isolation. In other words, they measure exactly the same thing.

(Contrast the other major predictor, high school GPA, where the predictive value of considering GPA + SAT in combination somewhat exceeds the predictive value of either metric individually.)

There is another implication as a consequence of the statement "the influence of test prep, cramming, test coaches, etc. is minimal for the subject tests." That is, I first read thag the students don't suffer under these conditions. It is a valuable trait no doubt, to be able to cram swaths of loosely associated facts. I'd argue that it's a vital trade for study, but perhaps it is less severe than only twenty years ago, because very powerful memory aids have become ubiquitious.

> predict performance about equally well

having no tires or no engine predicts performance of a car--or rather the lack thereof--equally well. Yet grip and horsepower are independent variables. I think that means SAT scores don't predict success too well at all beyond a certain threshold.

Having the right motivation (haha, a pun) for a certain disciplin might make a huge difference. So you can test e.g. vocabulary learning in general, or top9cal knowledge, which requires precise choices of vocabulary nonetheless, but one not found in a general dictionary. It's more like knowing which dictionaries exist, and what texts are referenced therein.

> I think that means SAT scores don't predict success too well at all beyond a certain threshold.

Try reading about it.

https://libres.uncg.edu/ir/asu/f/Webb_RM_2001_Top.pdf

> There is another implication as a consequence of the statement "the influence of test prep, cramming, test coaches, etc. is minimal for the subject tests." That is, I first read tha[t] the students don't suffer under these conditions.

This is not a valid inference to draw; the influence of test prep, cramming, coaches, etc. is also minimal for the main SAT, but students suffer through them anyway.

> Who cares what the conventional wisdom says? The psychometric results are that SAT I scores and SAT II scores predict performance about equally well in isolation and don't have more predictive value in combination than they do in isolation. In other words, they measure exactly the same thing.

This is point where you take a step back and conclude that if this is your measured result, you may have been measuring nothing at all.

Did you miss the first part?

> SAT I scores and SAT II scores predict performance about equally well in isolation

They're both good predictors. They're just the same predictor twice, as opposed to being two different predictors.

>The conventional wisdom has long been that the SAT subject tests are MORE predictive of future success at MIT, because the influence of test prep, cramming, test coaches, etc. is minimal for the subject tests.

Citation needed? The admissions office has much better data on this stuff, but in my experience, MIT students who performed well on these tests did so because their schools offered AP exams that were relevant. For students like me, we were S.O.L. and had to teach ourselves a year's worth of test material entirely by self-study while still maintaining top grades in school, doing research, and studying for the SAT.

To clarify: not disputing that there's less cramming, just that it's a better predictor.

I'm assuming most everyone taking the SAT subject test is also taking AP tests. AP tests were significantly more rigorous than SAT subject tests and would have provided more useful information to admissions. It took more prep to get a 5 on an AP Chemistry test vs a top score on the SAT subject test.

Some rich trustee's dumb kid probably got "screwed over" by a poor kid who put in insane hours. I dont see how this leads to egalitarian access to education given that on every other metric spoilt idiots carrying rich parents DNA will have an advantage.

If the reason is benign, I'm guessing it's just because AP tests have gotten so watered down that they're just a stand-in for the SAT subject tests at this point.

Reply on news.ycombinator.com