That is hilarious!
By the way, there is another intersting never ending lerner which is based on random images from the internet: http://www.neil-kb.com/
eg. A human might not be expected to know too much medical terminology, hence could easily get names/locations/functions of muscles and organs mixed up. Whereas a computer might be brilliant at that, but then suddenly not know what a door knob is (door handle? door knob of butter? etc...)
So ultimately to make a computer seem intelligent the problem is two-fold: Getting a high percentage of right answers, whilst also getting the "right" wrong answers.
Afterall, we don't criticize a human too much for not knowing something like the exact geographic location of a small African nation. But we definitely would if they thought that it was on Mars! It's exactly the same with computers, they're just far more likely to make wildly incorrect mistakes than understandably incorrect ones. So finding a way to mitigate computer mistakes, to make them milder, is every bit as important as eliminating them entirely - a potentially impossible task.
Whereas the uncontrolled randomness of computer mistakes could be horrendous. Hence why I'm saying an interesting avenue to look at in AI, might be to achieve acceptably incorrect answers (acceptable negatives), rather than just aiming for high true-positive and low false-positive rates.
Seems like they should run multiple instances against different but overlapping corpuses and see what they agree on. For all we know, any single person would forget what a door knob is if they had all the information in the world in their brain.
But maybe it already works that way internally, I don't know.
One could argue that humans have a similar problem of errors propagating. Areas that we feel strongly about can bias us against learning in fields such as religion, politics, and, of course, programming language design.
I think that it is possible that humans would make some similarly poor responses in areas that we don't talk about often.
Yet, the machine seems to come from an entirely differernt direction. It has much more facts accumulated than a child and can articulate and process them to absolute precision.
My point was more that perhaps we're a bit too focused on the wrong metric for success. The current criteria set is {positives, negatives, false positives, false negatives}, and we try to optimise for high/low degrees of one or the other in order to determine whether a particular approach is successful or not.
What is then overlooked, is that perhaps we don't need to have a near-perfect positive rate, but instead achieve an acceptably-incorrect false positive or false negative rate. Where the answer may be wrong, but it's not too far wrong. Much like a human might pin a country like India in the wrong place on the map, but wouldn't ever put it in the middle of the Indian ocean.
In summation: Perhaps the key for computers to appear intelligent, is not to be perfectly correct, but to be not too disastrously incorrect.
Have you spoken to some members of the general public and asked what they think? People believe all sorts of wacky things. We are just a lot more tolerant of the mistakes people make. http://www.buzzfeed.com/mjs538/things-americans-believe-in