It has truly begun, then. I've mistaken a real human for an AI.
marks the date
marks the date
The structure of that test (this was a while back - couldn't do an unconstrained one) had a panel composed of some chat bots and some humans (10 total, though the number isn't too important). Each was limited to the area of their specialty - I think there was a chatbot that had its specialty of bartending drinks.
The people evaluating them rated them on a distinct 1-10(?) scale. The question was trying to find a "what is the most human like chat bot."
Part of it was that the human expert in Shakespeare was considered to be more computer like than many of the chatbots in part because of the depth of knowledge in that domain.