Hacker News Reader

Comment by modeless | Hacker News Reader

A few short years ago we were eons away from passing the Turing test.

We're still eons away from passing the Turing test, we just keep lowering the bar for what the test is because some people spend too much time on the internet and want Her to be real too badly. This is a conversation I try every time a new version of some LLM comes out: https://chatgpt.com/share/eee38567-99d2-4dd2-a781-08b297e86d...

Why does that conversation fail the Turing test?

Granted there are some people on earth who can speak those 3 languages fluently, but it greatly diminishes the pool of available people. If you wanted to take the example to the absurd, you could just start listing more languages, and ChatGPT would happily spit out the answer translated to 100 languages in a few seconds. No person could do that.

ChatGPT is not trained to pass the Turing test. It is trained to be as superhuman as possible. I have no doubt that OpenAI could train a system to pass the Turing test within a year if that was their objective. In fact it seems like an anti-objective for them.

Now that's not to say that such a system would be undetectable by any possible adversarial technique. The Turing test is not unambiguously defined, but my definition would have it conducted with average well-educated people not specializing in AI and not having special knowledge of adversarial techniques for the specific machine being tested.

I’d say it goes very much against the spirit and idea behind the Turing test to fail an AI for having superhuman abilities it fails to hide.

As much as I admire Turing, I don’t think this particular idea of his was a sound one- an intelligent computer is not human, and lacks a human brain. It should be expected to be better at some things, worse at others, and overall noticeably different… even long after it exceeds human intelligence levels.

I think Turing's idea was profound and prescient honestly. The reason he put emphasis on deception and imitation in the test is because he wanted to know whether we had build a thinking machine, or just a tool. As you say, superhuman intelligent computers are not like humans and it's indeed not right to require that from them if we just want to build a knowledge repository. But if you want to know whether we've build a mind, his conception matters. To deceive, you have to have the imagination of what it's like to be you imagining what it's like to be me. You need a model of someone's cognition and your own, in real time while you interact with the world. And just architecturally that's not the case with LLM's. They don't have cognitive processes going on as they're interacting with you, they can't even in theory beat this.

It's actually scary to me just how smart Turing was because he immediately recognized, at a time when computers barely existed, that the real test isn't if a machine can play smart but if it can play dumb. Because that eliminates all the parlor tricks and requires actual reflection. Just mind blowing to anticipate that in 1950.

You are making the mistake of confusing a mind with a human mind… a vastly different type of mind should not be expected to be good at everything we are good at, even if generally far more intelligent. I think people generally make the same mistake with discounting the more intelligent animals by focusing on their inability to act human, and ignoring or not noticing that they can do things we are incapable of- like a dog navigating spaces in total darkness from scent alone.

A human mind, for example is not sufficiently generally intelligent to convincing mimic even a fish mind- we cannot generally emulate minds structured differently from ours- e.g. humans cannot actually do the meta task your concept of the Turing test expects AI to do. Even an actual human mind with vastly different experiences- simulated perfectly in a computer or even just a regular person from a very different culture will fail your concept of the turing test, because our mind is shaped by our experiences.

Also, I think your conception of LLM limitations is mistaken. We don’t fully understand them in theory- but I think the way to think of LLMs is that they are a probablistic world model able to simulate almost any scenario if setup properly. They are not themselves a mind, but if sufficiently good, they will simulate minds- and those simulated minds are actually minds but not human minds. Noam Chomsky published an article arguing how LLMs were incapable even in theory of doing things GPT4, which was released shortly after his article can do- showing that either his model of what they are, or of what is required for those tasks is mistaken. Ironically his way of thinking involves failing to understand that intelligent minds vastly different than human minds are even possible- while expecting AI to be able to not just understand but correctly navigate that situation.

Are we not still? How can you tell?

What?

There has been some odd people fooled by computers since the 70s, and 10 years ago we had competitions where the best hint people claimed they had was if their interlocutors were good at math or not.

It was clear 10 years ago that the Turing test would be beat soon.

My point is really that a lot of people thought we were eons away. "It was clear" to you, but your opinion was not shared by the majority probably until at least GPT-2 at the very earliest (2019). (Note that the sibling comment to yours still doesn't believe we are close...)

Similarly, today a lot of people think we are eons away from useful general purpose robots, but it is clear to me that they are coming soon.

That we'd beat the Turing test soon was a majoritarian opinion around here.

Honestly, I wasn't even that well informed to make my own. I kept postponing looking at the chatbots until after it was beat.

I disagree. 10 years ago there were people who thought that but they were far from the majority.

Reply on news.ycombinator.com