Hacker News Reader

Comment by dev_dull | Hacker News Reader

Goodhart's Law: “When a measure becomes a target, it ceases to be a good measure.”

SATs will quickly become one of the poorest indicators for success.

There’s an entire academic field designed around making reliable tests with valid results for psychological measurements, called psychometrics. IQ tests and we swear it’s not an IQ test like SATs, the GRE, GMAT, LSAT are all designed to be as close to ungameable as possible and it works. Practice effects exist, you can get better by doing more of them but they’re also limited. Usain Bolt is never going to run 100m in 5s and I’m never going to run a sub 10s 100m no matter how hard I practice. Likewise SATs are not going to stop predicting success in college or in life because they measure g, the general factor of intelligence.

https://en.wikipedia.org/wiki/Psychometrics

https://en.wikipedia.org/wiki/G_factor_(psychometrics)

Except that SATs measure something real (intelligence), so unless outright cheating and bribery are taking place, it will continue to be a good measure.

Citation needed. The huge amount of time and resources spent preparing for the SATs imply that it is not a direct measure of intelligence.

Is a someone who scored a 2300 after years of test prep necessarily more intelligent than someone who couldn't afford test prep or didn't have access to it and still scored a 2000?

See: https://nepc.colorado.edu/sites/default/files/Briggs_Theeffe...

"When researchers have estimated the effect of commercial test preparation programs on the SAT while taking the above factors into account, the effect of commercial test preparation has appeared relatively small"

In my experience, SATs measure nothing - only how well you can take the SAT, after taking a prep course my score went up like 200 points. One of my friends got a perfect score on the ACT after doing alright on the SAT. Some colleges are no longer taking it into account because everyone knows how BS it is.

EDIT: and to make it worse the college board charges exorbitant prices for everything involved in the process - it’s ridiculous

ACT should already be like that. I looked at the last one they sold, and, at least for math, you can easily game your way up to a 22ish on it. Which, granted, isn't needed for higher levels, but that's above what some states set as their benchmark, so...

I'm sure the other sections are the same, though I didn't look at those. But, really, any math exam with multiple choice answers can be gamed, at least to an extent. And then people will start teaching how to game it (especially if funding comes from them, which some does in the state I teach in)

Not sure that holds up under all cases. Grades, as an example, are both measures and targets.

Reply on news.ycombinator.com