Hacker News Reader

The paper does present an "Average American" comparison when it refers to Amazon Turk. The Turk workers performed significantly better on middle-school reading comprehension (RACE-m) while Gopher performed slightly better on high-school reading comprehension tasks (RACE-h).

It might be fun for a laugh.

What actual value would an AI that produces answers similar to the average person have, though? Non-expert answers for interesting questions are pretty much meaningless -- the whole point of an advanced society is that we can avoid knowing anything about most things and focus on narrow expertise.

Probably no value. I was interested in a comparison point, that is all. You can't understand how far away you are from something unless you measure it. In other words, if I asked you: how does this compare to the average person, you cannot answer because this table didn't measure it.

Yea, we had to dumb it down to compete in a Turing test. Otherwise it was unbelievable.