Think of it instead as an equation solver, with the initial condition X=4. These tricks are ways for user input to set X=9. They’re more akin to SQL injection than comprehension of incentives and policy.
Person A: If we keep our food in this frozen ice igloo then Nubiroakox, the God of Pestilence will spare us and not ruin our food.
Person B: It's actually not Nubiroakox, it's just that the conditions for bacteria to grow depend on temper...
You: If attributing this behavior to magic a) gives practical insight into it's behavior that b) also happens to result in the behavior you would expect, why not do it?
It's more akin to giving your Roomba a name, and describing it's behavior in terms of "it likes to eat goldfish, but not string because it gets tangled up".
That's still anthropomorphizing, but it's not ascribing anything magical or outright wrong to it.
It can be a useful and cute fiction, as your Roomba example, but it leads to bad decision making (“I’ll put more goldfish on the floor to make the Roomba happy”).
I guess it all comes down to how important accuracy is to you in a particular context. I anthropomorphize my dishwasher (it HATES wine glasses), but I don’t make professional judgments about dishwashers.
We do that with a lot of things every day and a perfectly capable of drawing the lines when things come down to it.
On a more philosophical note. If we some day end up creating AGI, having anthropomorphized it will actually be to our benefit.
So you are technically correct but this is not about being technically correct.
The only controversy is whether and anthromorph is an isomorph of an accurate model of ChatGPT. I doubt that it is, yet is close enough to fool a lot of people, then surprise them at random.
Because both of those steps are really unreliable. The insight we receive is only as useful as the behavior it corroborates, and that behavior is only as notable as the patterns it follows. AI... doesn't really behave consistently. Assuming that it will follow the behavior you expect ignores the actual motive of the model, which is to fill-in the blank after whatever you typed.
Personally, I think it's harmful because anthropomorphic AI is a poor abstraction for text-completion models. That being said, most people don't ever seem to know the difference.
This is a classic example from philosophy of mind. It doesn't completely harmonize with the question you're asking, but it concisely summarizes two opposing views.
What's my view? That we do it all the time as a matter of course. "Why not do it?" isn't the question. The question is how much weight we should give to the result, and whether we should take care to question the results and try something else, too.
Something closer to the term "GPGPU" would have at least given more of a sense of reality. There is nothing intelligent about a GPU crunching billions of numbers in a massively parallel way and converting those numbers into something that happens to interface convincingly with the human sensory systems. It's just an algorithm with so much data that one can't comprehend thinking about the sheer size of it, so naturally it must be thought of as "intelligent."
It wouldn't be a ChatGPT HN thread without someone claiming that LLMs are stupid predictors incapable of "real" understanding.
Of course ChatGPT comprehends things. It does so under any useful definition of the word "comprehend". The "grokking" paper [1] shows that it learns fundamental principles of various algorithms and isn't just predicting text. If this isn't comprehension, humans aren't capable of comprehension.
[1] https://arxiv.org/abs/2201.02177 (check out the citation list too)
> If this isn't comprehension, humans aren't capable of comprehension.
All I'll say is that conflating human and machine intelligence is exactly what people are trying to stop. ChatGPT is not a human mind, and LLMs in general are a poor analog for human intelligence. This sort of "AI convergence" mindset will set you up for the most disappointment of anyone getting invested in the tech. We'll be lucky if it can summarize emails reliably enough to sell as a product.
I don't think we understand human intelligence nearly well enough to make this claim.
Personally I think that what we consider our "conscious" part is somewhat defined by what we can put into words, and it is in the end putting one word after another.
If I asked an AI to respond to this comment, it might write an impassioned defense of itself one-word-after-another, but it wouldn't necessarily base it's defense on freestanding logic. It's foremost goal is writing coherent text, not being right or wrong or minimizing human risk. Paving over any of those assumptions is a fun thought exercise, but doesn't reflect the nature of the technology at-hand or even what we know about human consciousness.
How do you know that doesn't happen in some latent space?
> It's foremost goal is writing coherent text, not being right or wrong or minimizing human risk.
That's also true for human bullshitters, it doesn't mean that they're not intelligent.
If it does, then it contradicts what OpenAI says about their own product. GPT is an autoregressive model trained on text. It's sole purpose, as designed, is to complete text using stochastic reasoning based on the text it learned from. It seems incredibly unlikely (to me) that someone would deliberately mis-design such a system to be capable of independent thought.
If you have evidence of some additional processing layer, I'd love to hear it though.
> That's also true for human bullshitters, it doesn't mean that they're not intelligent.
Bullshit by any other name smells just as bad.
University students complete text using stochastic reasoning based on text they learnt from. Again, that doesn't mean they're not intelligent.
> It seems incredibly unlikely (to me) that someone would deliberately mis-design such a system to be capable of independent thought.
If you train something to produce convincing text then it's perfectly reasonable that it could develop these faculties.
> If you have evidence of some additional processing layer, I'd love to hear it though.
I don't think either of us can conclusively claim that a 175-billion parameter model either does or doesn't do something. I just think there's no particular reason it couldn't.
> Bullshit by any other name smells just as bad.
The question isn't whether GPT produces bullshit or not (it often does), but whether it's fundamentally incapable of thinking like a person. People bullshit too so I don't think that really proves anything.
I really don't know what to tell you. You're free to believe whatever you want, but the simplest feasible explanation is the science we used to make these models. At no point was integrating human-like thought a consideration of the model. Just human-like speech.
You might find the answer to be arbitrary or meaningless, but it is an important distinction. I could pretend to be a random number generator well enough to fool anyone who's distinguishing me from a computer. That doesn't mean that I'm a reliable source of entropy, or equally as random as the other, similar result. Treated as a black-box they are the same; treated as a mechanical system, they could not be more different.
It's like the aliens coming to our planets and saying we can't be intelligent because we're made of meat.
Does it HAVE goals? I don't think there is any evidence that it does. Even if it developed the ability to do all of those things you mentioned, it still wouldn't have any idea what it is doing, let alone why it is doing it, because it's just a language model. That is all it does. And that's why I don't quite understand how people can be so adamant that there is any question about whether it is conscious or not. This whole debate is a trap. It is a brute fact that it is not conscious. It does not have a goal of producing coherent text. That is the goal of the people who wrote it. If we treat the brain as a computer, then GPT lacks the subsystems/algorithms/whatever to do what brains do. It fails at many basic tasks that anything capable of general-purpose reasoning could do. It's often not apparent how spectacularly it can fail because the tasks are so mundane that we don't bother even asking it to do them.
You're right that the model doesn't "know" that necessarily, but it has been designed to produce convincing text when given a small amount of entropy to start with.
Well, of course--you just specified what the end is. I'm not trying to be a smart ass. This is just where u have to go to discuss this.
Consciousness isn't a part of us. "Us" is part of consciousness.
I have no idea what that means.
When people make a "conscious decision", or are "conscious" of something, it usually means something they can put into words as opposed to a vague feeling. Being able to communicate with language is what sets us aparts from animals. That's why I think language models are actually really powerful.
Just because one can inject an unauthorized command in a SQL statement, it doesn't mean the database is a flawed humanoid. This is just nonsense...
Protecting against SQL injection is trivial because commands are highly structured and user input is clearly identifiable.
Prompt engineering is more complex since it relies on natural language, which is a lot less structured.