Hacker News Reader

Comment by glial | Hacker News Reader

Shorter is good, but dynamic tests (called "computer adaptive tests" or CAT in the industry) make it difficult to follow longstanding best-practices for test taking. When I took the SAT, I could quickly scan through the questions in the current section and mark those that looked difficult, and save them for the end. With a CAT test, it's typically one-question-at-a-time, and if you're a strong test taker, they get successively more difficult. They are more efficient tests, but (IMHO) less pleasant to take.

View on HN

That is true for adaptive tests that operate on a per-question basis. What I have read of the new SAT is that it operates at a session level. So if you do well on the entire first session, you get a harder version of the next session on that topic (reading or math).

Students should still be able to use the technique you mention to identify hard questions and save them for last.

> What I have read of the new SAT is that it operates at a session level. So if you do well on the entire first session, you get a harder version of the next session on that topic (reading or math).

That is positive. I've seen issues with adaptive tests not being resilient to accidents and brain farts: if a kid fat-fingers the response to a question early in the test, they may never get the computer to give them questions of a suitable difficulty later in the test. Thus they never even have a chance to get a score that accurately reflects their ability.

If the SAT is session-based, each session needs to be large enough that a single question can't tank the whole thing, but at the same time there also needs to be enough sessions to allow for properly dialing-in the knowledge level. In reality, the ideal situation is that the test should allow for an unlimited number of sessions, but each session is time-limited. Something along the lines of stopping once the test-taker gets below a certain score on 2 consecutive sessions.

> If the SAT is session-based, each session needs to be large enough that a single question can't tank the whole thing

I think each session is either half or a third of the test, for that subject. There could actually be more than two versions of the later session, to ensure that a student who makes a mistake or two isn't prevented from getting a relatively high score.

> make it difficult to follow longstanding best-practices for test taking.

Isn't that good? The effectiveness of test-taking strategies is a (minor) flaw in the SAT. People taking an ideal test would not benefit at all from learning strategies.

(The flaw is minor because all our methods for assessing people are gameable to some extent, and standardized general tests like the SAT are among the least gameable.)

Some of the math competitions I participated would penalize for skipping around. You were awarded +5 for correct answers and -4 for incorrect or skipped questions. It was not uncommon for negative scores. In one test, any stray marks were also marked as incorrect. No erasing, no changing a 7 to a 9, or anything was allowed. The questions leaned toward an increasing difficulty, but there could something very difficult followed by a string of much easier questions. So additional math had to be done to see if it would be better to stop or skip.

I don’t see how this is any different from a normal test? If you skip a question or get it wrong the penalty is you don’t get any points for it.

As stated: "You were awarded +5 for correct answers and -4 for incorrect or skipped questions."

Answer question 1 correctly: +5 Skip question 2: -4 Answer question 3 correctly: +5 Net score: +6

If skipped were just ignored, the net score would be +10.

Interesting, I don't think I've ever seen a test like that. What was it?

Again, as stated: "Some of the math competitions"

Don't know how wide spread they are in other states, but it was part of the UIL Academics teams as statewide competition between school districts in Texas.

It's still equivalent to "normal" tests, because the cost of skipping and the cost of answering incorrectly are the same. A test in that format with N questions and a test in the normal "1 point for a correct answer, 0 for anything else" with N questions are related as follows, assuming C correct answers:

  Score in the +5/-4 system: 5 C - 4 (N-C) = 9 C - 4 N
  Score in the +1/0 system:  1 C + 0

It's basically a "9 points for a correct answer, 0 for anything else" test that simply starts with a score of -4 N.

It's very similar to the difference between Celsius and Fahrenheit.

how many normal tests would ever result in a negative score? it's quite devastating to any shred of morale one might have. test had 80 questions so a max score of 400. i personally never witnessed someone receiving anything over 300.

you're really trying to make something into something it is not.

for the curious, here's an example test: https://bryantheath.com/files/2021/02/Q104.pdf

edit: you only get 10 minutes to take the test

> It's basically a "9 points for a correct answer, 0 for anything else" test that simply starts with a score of -4 N.

But it's not. Skipped questions before your final answer get a bad score, while skipped questions after your final answer get a medium score.

If you take a blank test and answer the last question only, your score drops by hundreds of points.

No normal test does that. That's how it penalizes skipping around.

Number Sense! Despite how pointless those tests were, the whole experience was super fun!

Howdy fellow Texan! I still use some of the "tricks" learned to do that test. However, I struggled for a long time in my higher math classes for not having any work to show as I was just doing it in my head. It was finally solved when told that the AP exams gave partial credit based on the work shown.

I also did the Calculator tests. It's why I learned to 10-key.

You're right - I still use number-sense tricks for fast multiplication :)

For Calculator, I wrote https://git.io/ti84rpn so that I could use the fast parenthesis-less Reverse Polish Notation without having to adapt to a whole new calculator keyboard.

It's cute but doesn't make any difference. Add 4 to the score for every question and you get 9 or 0. The factor of 9 makes no difference so call it 1 or 0. Now your score is just the number of questions you got right, but the ranking is exactly the same as the original scoring system.

Again, another person responding with the same trope of a response totally missing the fact that assigning 1 or 0 means an always positive score. You will never know the shame of receiving a negative score. It's part of the competition whether it was designed that way or not, it is exactly what teenage boys have turned it into. The concept of a negative score is a pretty good motivator.

It depends if skipping means questions not answered or if it means questions skipped to answer later questions. But either way it's more like playing gameshow host than making a meaningful exam.

Reply on news.ycombinator.com