Hacker News Reader

Comment by toddh | Hacker News Reader

"But I wouldn't call this unfair...precise timing just happens to be one thing computers are better at than we humans. It's not like I think Watson should try buzzing in more erratically just to give homo sapiens a chance."

Wouldn't it be better to build a distribution of champion level response times and have Watson draw randomly from that distribution? That would seem more fair.

View on HN

Here's a crazy idea... what if we changed the rules to Jeopardy so that if the buzzer presses are within a 50ms window then randomly select a contestant.

I think this would be a good change for even all human games.

I think that's a great idea, allow for different pools of time after the buzzer, 0-50ms, 50-100ms, etc and if you end up in the same pool as another player there is a random draw for who gets to answer first.

Seems to me they should let people click their buzzers as soon as they want to, and then just take whoever clicked first. Having to wait for the question to finish being asked seems like it makes it a lot more random rather than skill-based.

It used to be this way, didn't it? It seems to me I remember people used to often ring in with an answer before Alex was done reading the question.

According to Wikipedia: "Before Trebek's second season, contestants were able to ring in at any time after the clue had been revealed, and a buzzer would sound whenever someone rang in. According to Trebek, the buzzer sound was "distracting to the viewers" and sometimes presented problems, as contestants would inadvertently ring in too soon, or ring in so quickly that by the time he finished reading the clue, the contestant's five-second limit had expired. He also said that, by not allowing anyone to ring in until the clue was finished, home viewers could play along more easily, and faster contestants would be less likely to dominate the game."

http://en.wikipedia.org/wiki/Jeopardy!#Ringing_in

The first rule of competitive game playing is that anything in the rules is... game.

Illustrating that the computer has the advantage over humans is the entire point. Competitive game play is not about 'fair,' in fact, the best players specifically try to make the game as unfair as possible, within the rules of the game.

We already know that computers have an advantage over human reaction time. We don't need Watson to tell us that, my laptop can press a button faster than I can. This is like requiring each player to run a mile before being able to answer the question... and then letting one guy drive a Corvette instead.

And if the challenge IBM set for itself was "can we build a machine that travels faster than a person", that would be perfectly legitimate too.

Of course, the impressive thing is that Watson can answer the questions at all.

The point being that at the highest levels of play, all of the contestants know most of the answers. This is a very important thing to remember. It means that even in all-human games, it's not "do you know the answer" but "can you ring in fast enough."

So it still is really quite impressive that reaction time matters at all to Watson, because it implies that it can compete at the highest levels.

But the rules have already been bent to help Watson in areas where it's weak:

• receives question (and answers) as coded text, rather than having to sight-read or listen to them

• no audio/video clues

• no categories that require an extra Trebek-explanation

It may even be receiving a direct signal when it's OK to ring in, rather than assessing Trebek's cadence and the same light-indicator as the human contestants rely upon.

So other 'balancing' changes shouldn't be out of the question.

I particularly like the idea that the penalty for a moment-too-early buzzing could be eliminated, and/or treating all buzzes in a small window around the traditional earliest-allowed moment as simultaneous. That still rewards quickness, without heavily advantaging the computer's ability to perfectly synchronize with the buzzing-period.

Once two buzzes are considered simultaneous, the contestant chosen to answer could rotate in round-robin fashion, or (for maximum drama) be the one furthest behind.

The easiest and fairest fix would be to hook Watson's buzzer up to a dataset of previous Tournament of Champion question-winning buzzer times. Or just use Ken and Brad's previous buzzer times for even more control. Pick a new human-achievable reaction time for every question.

AFAIK, Watson was just using the light signal to decide when to start buzzing in. Just the light, ignoring when Trebek was going to finish saying something.

Presumably a human could consistently beat Watson to the buzzer by predicting when the light would go on.

Not sure about 'consistently', though trying to predict the light seems a reasonable strategy to attempt.

There's a lock-out that applies if you buzz too early, and the variance of the 'beats' between Trebek's finish and the light, if that's a human-mediated process, may be far larger than Watson's light-fired reaction time.

There's a whole book on the concept of game balance, focused on arcade fighting games (Street Fighter), written by David Sirlin, who balanced Super Puzzle Fighter II Turbo HD Remix and Super Street Fighter II Turbo HD Remix.

http://www.sirlin.net/ptw

http://en.wikipedia.org/wiki/David_Sirlin

Particularly relevant is the section on 'scrubs', the people who arbitrarily ban certain techniques for being 'cheap' instead of coming up with ways to counter them:

http://www.sirlin.net/ptw-book/intermediates-guide.html

As he works explicitly in the world of game balance, he acknowledges that it must always be reasonably possible to counter all usable techniques, or else the game is flawed. He even advocates banning techniques that honestly can't be countered, as a way of turning bad (unbalanced) games into good (balanced) ones.

http://www.sirlin.net/ptw-book/what-should-be-banned.html

So I suppose the upshot is that Jeopardy!, as it is now, allows any reaction time physically possible given the limitations of the buzzer and the constraints of the enforced time limit (which penalizes you if you attempt to buzz in before Alex finishes, or for approx. 0.2 seconds after he finishes). You could imagine a Jeopardy! variant with different rules in that respect, but it would not be the same game.

Reply on news.ycombinator.com