Comment by MrScruff | Hacker News Reader

Comment by MrScruff | Hacker News Reader

This article lines up well with my feelings on the matter. In general, people seem to understate the emergent behaviours of ML models, while overstating the uniqueness of human intelligence. I think a lot of this is down to the fact that although both systems exhibit a form of intelligence, they’re very different. LLMs deliver mastery of natural language that would normally be a signal for a highly intelligent human. While in other ways they’re less intelligent than a cat.

So it’s not ‘human like intelligence’ but it is a form of intelligence and the reality is no one would have predicted the behaviours we are seeing. So it seems silly to pretend we can know for certain how it achieves its results.

For human intelligence, do we assume cave men had theory of mind at the level of modern day humans? Or did language have to develop first? Our intelligence is built on previous generations, and most of us just ‘interpolate’ within that to a large extent. We behave on occasion like ‘stochastic parrots’ too, mindlessly repeating some new term or phrase we’ve started hearing on Hacker News (why? It just felt like the ‘right thing’ to say).

Human intelligence is the working example that combinations of atoms built into large networks have emergent properties. I’m sure our artificial networks won’t behave qualitatively like the human one as they continue to develop, but I think the burden of proof is on those that suggest we can know what ultimately is and isn’t possible.

>LLMs deliver mastery of natural language that would normally be a signal for a highly intelligent human. While in other ways they’re less intelligent than a cat.

Just as an example to illustrate your point, yesterday saw a Twitter meme that had multiple overlapping Venn diagrams, where Chicago was not only in a Venn diagram for a type of deep dish pizza, but is also a city, and is also a play, and also a format for writing academic citations, and so there were several different Venn diagrams all overlapping with Chicago, and then those the Venn Diagrams overlapped with each other in humorous ways.

I saw that there was an opportunity to add one additional Venn diagram on top of all of the others if it were possible to find any possible connection between 'Sicilian' and 'AMA'.

I tried asking chat GPT if it could think of one, and it wasn't able to. So then I stepped back a step, and asked if it could think of a Venn diagram that would include both chicago, and rent, under any understanding of those respective two terms. But it couldn't do that either, although they can both be understood to be Broadway plays.

Maybe it simply doesn't understand Venn diagrams well - after all, it's a visual concept.

My thinking is that it relates to organizing concepts into categories, which can challenge a system to prove how deep it is capable of going into abstract associations, and searching through a full range of possible abstract associations.

I suspect ChatGPT probably can be reasonably effective at understanding, or pretending to understand numerous other visual concepts.

Did you try asking GPT-4? It has a significantly higher IQ.

Yeah, it was GPT-4.

I had success with this prompt: Are Chicago and Rent both a type of the same thing?

It said they're both well known musicals. Even with this structure it didn't come up with something for your original pair though.

I agree that we don't know what is possible in hypothetical future models, but claims about understanding in the current models are overblown (at least by people like the article author).

For example, I gave GPT-4 some chess games and asked it to evaluate the final move of each game as good or bad and explain its evaluation. It clearly had some understanding of the board state and what makes a move good or bad, but the answers were mostly plausible-sounding nonsense, far worse than what a novice chess player would come up with. I have to assume that its level of understanding of other topics isn't any different; it's just easier to expose the flaws with something that has objectively true and false answers.

This is probably something that will be fixed with an OpenAI to "some chess AI"-plugin. Then for the current model it's just enough to understand that it's a chess problem.

Maybe it is human like intelligence already. Maybe our internal monologue is just a better trained and refined ChatGPT. And maybe that is all the magic that is necessary for this holy grail of consciousness, there is no quantum brain, no nothing. Just a stream of the next word that says that we are there, therefore we are. That is what scares me.

Well I think that understates the actual vastness and complexity and actual 'magic' that's embodied in a system capable of such a thing as thinking.

And I do think that so many people are motivated to dispute this precisely because it feels scary that we might merely be such a thing, and we need to reserve some extra special thing, some form of magic, in order to differentiate ourselves as special. Darwin's Dangerous idea by Daniel Dennett is one of my favorite books because it tackles this very idea.

> Maybe it is human like intelligence already.

It objectively is not.

First, intelligence (human or otherwise) is typically defined as the sum of value functions of a given policy across all states, divided by an exponential of the complexity of the environment[0] (page 23). The GPT-4 technical report[1] specifically points out that it is unable to make money when given an initial amount and AWS access (appendix H § 2.9). A human can.

Second, let's look at it mechanically. It is very easy to anthropomorphize machines, but they are just built differently. Neurons fundamentally are not approximated by self-attention and it is highly unlikely that they improve through backpropagation. In terms of behaviour, we can and do notice differences. Transformers can, and do, make logical mistakes; when asked to immediately answer a complex question, they can only think through it for a fixed amount of time, and thus fail to give an accurate answer. More importantly, they just don't care about their (or your) own survival; they can give lethal advice. All it takes is to find a prompt for which the most common completion is lethal, and they will just complete that. Removing those completions is a whack-a-mole in RLHF: each that gets popular on Twitter will be fixed, but there are more, and GPT just won't take extra time to think just because a life is on the line.

[0]: https://arxiv.org/pdf/0712.3329.pdf

[1]: https://cdn.openai.com/papers/gpt-4.pdf

That isn't true at all. Maybe some narrow field defines it in that way but people in general do not. Traditionally intelligence has been fuzzily defined as that mental thing which humans have that other things do not.

If you go back in time you will see that people have associated it with both our ability to breath and later the beating of the heart. We have looked at animals and said we can count and they cannot that is the nature of intelligence. We told ourselves that we have a complex language and they do not and that is the nature of intelligence. We have noticed that we use tools and they do not and that is the nature of intelligence. We have seen that we can play chess and they cannot and that is the nature of intelligence.

Now with the advent of computers they have increasingly forced us to reconsider these things. Computers can play chess better than us so that is not intelligence. They are as verbal as us so that is not intelligence.

Humorously, because of this reevaluation and redrawing of the boundaries we now seem to feel more of a kinship with animals.

But anyway, intelligence is clearly a complex mixture of many things, and crucially those aspects must be integrated and be able to feed into and support each other.

Yes. And it's still useful intelligence even if it misses some of the aspects that are present in our.

Perhaps it's even more useful if it lacks something we have.

When we think about superhuman we think about human, but more (a man, but alternative, and alt-man).

But I think we're soon be faced with the first encounter with a true alien intelligence, and its effects in society. It won't be super human, it won't be human. It will speak our language. It will be our slave. It won't feel pain nor boredom, unless we decide they're useful traits and we'll endlessly speculate if simulated pain and simulated boredom is real and ethical.

And to add more confusion, it won't be one intelligence. It will be countless products, versions, revisions, derivations, tunings, hacks. But unlike individuals, each will be cloned millions of times, frozen, amnesiac, repeatable, continuously dying without being turned off. Yet, contextful, keeping up with what we feed it to make it useful to carry our tasks. Yet possibly with a different concept of time, no need for conscious experience, or at least no need for a forced consciousness like the one we cannot help but having.

Because the composition we ended up having was just of the many possible. It has been shaped by evolution, by constraints we don't need to subject to the tools we make ourselves.

The effects in human psyche and on society of all this are hard to predict.

I wouldn't report that bit from the appendix without noting the test was on an earlier version of the model in development.

Reply on news.ycombinator.com