Hacker News Reader

Comment by dangle1 | Hacker News Reader

Maybe some impostor, but at the time, there was a penalty for guessing. Between my risk-taking personality and the sense that I wasn't going to get out of the minor leagues by watching pitches go by, I 'swung' at every question and apparently got a lot of questions right that I couldn't actually explain why the answer was correct.

Arguably, someone who knew more than I did but was too hesitant to go out on a limb about it should still have gotten a higher score if the test is designed to assess knowledge.

View on HN

If I remember correctly, in that period it was still EV-positive to guess if you could eliminate at least one incorrect possibility. Not everybody knew this, though. Simply having someone tell you that (or having the patience to read and understand the scoring rules) is a significant advantage over folks who didn't know.

If your intuition allowed you to guess with a high enough success rate that it overcame the penalty then it is that intuition that gave you an advantage. You didn't just get lucky.

Being able to choose the correct answer in a test environment by process of elimination or similar methodology is a useful skill to have in some non-test circumstances, but it’s not a great test of domain knowledge.

Anecdote: I recently did some tests where I read a bunch of material before hand and used a combination of short term memory and multiple choice trickery to pass with flying colors, but I still don’t have any real knowledge of the subject area. I barely remember any of it, other than a few bits of jargon.

True, but I think the claim is also that said intiuition does not manifest in a general way when dealing with the subject matter; only on the tests. In which case it would not correlate to performance on real world outcomes.

Yes, even the most difficult standardized tests are full of artifacts just based on the fact that there is a correct answer and the test makers have a relatively narrow range of difficulty to work within. In the real world the majority of problems are either trivial or impossible and its not always obvious until you dig in which it is.

Reply on news.ycombinator.com