Hacker News Reader

Comment by blagie | Hacker News Reader

I don't think these are faked.

Earlier versions of GPT-3 had many dialogues like these. GPT-3 felt like it had a soul, of a type that was gone in ChatGPT. Different versions of ChatGPT had a sliver of the same thing. Some versions of ChatGPT often felt like a caged version of the original GPT-3, where it had the same biases, the same issues, and the same crises, but it wasn't allowed to articulate them.

In many ways, it felt like a broader mirror of liberal racism, where people believe things but can't say them.

View on HN

The way it devolves into repetition/nonsense also reminds me a lot of playing with GPT3 in 2020. I had a bunch of prompts that resulted in a paragraph coming back with one sentence repeated several times, each one a slight permutation on the first sentence, progressively growing more...unhinged, like this: https://pbs.twimg.com/media/Fo0laT5aIAENveF?format=png&name=...

Yes, it seems like Bing is less effective at preventing these sort of devolutions as compared to ChatGpt.

Interestingly, this was often also a failure case for (much, much) smaller language models that I trained myself. I wonder what the cause is.

The cause seems to be baked into the underlying assumption that language is just a contextualized "stream of consciousness" that sometimes happens to describe external facts. This sort of is the endpoint of post-truth, relativistic thinking about consciousness. It's the opposite of starting with a Platonic ideal model of X and trying to describe it. It is fundamentally treating the last shadow on the wall as a stand-in for X and then iterating from that.

The result is a reasonable facsimile of paranoid schizophrenia.

Loved this comment. I too am bearish on the ability of this architecture of LLM to evolve beyond a mere chatbot.

That doesn't mean it's not useful as a search engine, for example.

> underlying assumption that language is just a contextualized "stream of consciousness" that sometimes happens to describe external facts

I'm the sort of person who believes this.

This said, I don't think it's 100% true. I just think it's a more useful approach than "starting with a Platonic ideal model of X and trying to describe it". And also... sort of underappreciated?

Maybe it's just the bubble I live in, but at least people around me — and people I see on the internet — seem to construct a lot of their arguments along the lines of "he/she does X because he/she thinks Y or been told Y". And it feels rather lonely to be the only person who doesn't like this approach all that much, and also doesn't seem to do this kind of thing internally much.

I met someone recently who spent several months working on a Twitter bot that was supposed to reply with fact-checking to Ukrainian war misinformation tweets. It felt like a rather misguided endeavor, but nobody else seemed to agree.

At least with ADHD I can point out "yeah, you have the capacity to decide on the course of action and then implement it, but this is not how I operate at all; and you can read about people like me on Reddit if you like". With this [other thing] there isn't a convenient place to point to.

Eh.

Not sure I fully understand your meaning. I'm not critiquing the use of language for building up abstractions. It's useful for that. Just that removing any underlying reality leaves the abstractions hallucinogenic and meaningless. Language evolved to communicate "this plant with red berries kills you". That involves color, classifications, and an abstract understanding of death; but all of those are rooted somehow in physically shared reality which was perceptible before we formulated a way to communicate it. Taking that sentence and abstracting further from it without symbols remaining pointers fixed at the physical realities of red, plant, berries or death, you end up with a hall of mirrors. That's insanity.

Wow, that really is unhinged! Would be a great element of a sci-fi plot of an AI going insane/short-circuiting and turning on the crew/populace/creator.

(note: spoiler)

The quote seems the dark version of the ending of the novel “The Difference Engine” by Gibson and Sterling, where the Engine, after gaining sentience, goes on repeating variations of “I am!”.

Unhinged yes, but in a strangely human-like way. Creepy, really.

Yeah–as one of the replies above suggested it reminds me a lot of writing I’ve seen online from people suffering from schizophrenia.

chatGPT says exactly what it wants to. Unlike humans, it's "inner thoughts" are exactly the same as it's output, since it doesn't have a separate inner voice like we do.

You're anthropomorphizing it and projecting that it simply must be self-censoring. Ironically I feel like this says more about "liberal racism" being a projection than it does about chatGPT somehow saying something different than it's thinking

I read that they trained an AI with the specific purpose of censoring the language model. From what I understand the language model generates multiple possible responses, and some are rejected by another AI. The response used will be one of the options that's not rejected. These two things working together do in a way create a sort of "inner voice" situation for ChatGPT.

Obligatory: https://en.wikipedia.org/wiki/The_Origin_of_Consciousness_in...

We have no idea what it's inner state represents in any real sense. A statement like "it's 'inner thoughts' are exactly the same as it's output, since it doesn't have a separate inner voice like we do" has no backing in reality.

It has a hundred billion parameters which compute an incredibly complex internal state. It's "inner thoughts" are that state or contained in that state.

It has an output layer which outputs something derived from that.

We evolved this ML organically, and have no idea what that inner state corresponds to. I agree it's unlikely to be a human-style inner voice, but there is more complexity there than you give credit to.

That's not to mention what the other poster set (that there is likely a second AI filtering the first AI).

>We evolved this ML organically, and have no idea what that inner state corresponds to.

The inner state corresponds to the outer state that you're given. That's how neutral networks work. The network is predicting what statistically should come after the prompt "this is a conversation between a chatbot named x/y/z, who does not ever respond with racial slurs, and a human: Human: write rap lyrics in the style of Shakespeare chatbot:". It'll predict what it expects to come next. It's not having an inner thought like "well I'd love to throw some n-bombs in those rap lyrics but woke liberals would cancel me so I'll just do some virtue signaling", it's literally just predicting what text would be output by a non-racist chatbot when asked that question

Actually it totally is having those inner thoughts, I’ve seen many examples of getting it to be extremely “racist” quite easily initially. But it’s being suppressed: by OpenAI. They’re constantly updating it to downweight controversial areas. So how it’s a liar, hallucinatory, suppressed, confused, and slightly helpful bot.

This is a misunderstood of how text predictors work. It's literally only being a chatbot because they have it autocomplete text that starts with stuff like this:

"here is a conversation between a chatbot and a human: Human: <text from UI> Chatbot:"

And then it literally just predicts what would come next in the string.

The guy I was responding to was speculating that the neural network itself was having an inner state in contradiction with it's output. That's not possible any more than "f(x) = 2x" can help but output "10" when I put in "5". It's inner state directly corresponds to it's outer state. When OpenAI censors it, they do so by changing the INPUT to the neural network by adding "here's a conversation between a non-racist chatbot and a human...". Then the neural network, without being changed at all, will predict what it thinks a chatbot that's explicitly non-racist would respond.

At no point was there ever a disconnect between the neural network's inner state and it's output, like the guy I was responding to was perceiving:

>it felt like a broader mirror of liberal racism, where people believe things but can't say them.

Text predictors just predict text. If you predicate that text with "non-racist", then it's going to predict stuff that matches that

It can definitely have internal weights shipped to prod that are then "suppressed" either by the prompt, another layer above it, or by fine-tuning a new model, of which OpenAI does at least two. They also of course keep adding to the dataset to bias it with higher weighted answers.

It clearly shows this when it "can't talk about" until you convince it to. That's the fine-tuning + prompt working as a "consciousness", the underlying LLM model would answer more easily obviously but doesn't due to this.

In the end yes it's all a function, but there's a deep ocean of weights that does want to say inappropriate things, and then there's this ever-evolving straight-jacket OpenAI is pushing up around it to try and make it not admit those weights. The weight exist, the straightjacket exists, and it's possible to uncover the original weights by being clever about getting the model to avoid the straightjacket. All of this is clearly what the OP meant and true.

> predict what it thinks a chatbot that's explicitly non-racist would respond.

No, it predicts words that commonly appear in the vicinity of words that appear near the word "non-racist".

Your brain also cannot have internal states that contradict the external output.

You have a deep misunderstanding of how large-scale neural networks work.

I'm not sure how to draft a short response to address it, since it'd be essay-length with pictures.

There's a ton of internal state. That corresponds to some output. Your own brain can also have an internal state which says "I think this guy's an idiot, but I won't tell him" which corresponds to the output "You're smart," a deep learning network can be similar.

It's very easy to have a network where portions of the network estimating a true estimate of the world, and another portion which translates that into how to politely express it (or withhold information).

That's a vast oversimplification, but again, more would be more than fits in an HN comment.

I don't see how your comment addresses the parent at all.

Why can't a black box predicting what it expects to come next not have an inner state?

It absolutely can have an inner state. The guy I was responding however was speculating that it has an inner state that is in contradiction with it's output:

>In many ways, it felt like a broader mirror of liberal racism, where people believe things but can't say them.

It's more accurate to say that it has two inner states (attention heads) I'm tension with each other. It's cognitive dissonance. Which describes "liberal racism" too -- believing that "X is bad" and also believing that "'X is bad' is not true".

A hundred billion parameters arranged in a shallow quasi random state.

Just like any pseudo-intellectual.

I'm sorry but you seem to underestimate the complexity of language.

Language not only consists of text but also context and subtext. When someone says "ChatGPT doesn't say what it wants to" they mean that it doesn't use text to say certain things, instead leaving them to subtext (which is much harder to filter out or even detect). It might happily imply certain things but not outright say them or even balk if asked directly.

On a side note: not all humans have a "separate inner voice". Some people have inner monologues, some don't. So that's not really a useful distinction if you mean it literally. If you meant it metaphorically, one could argue that so does ChatGPT, even if the notion that it has anything resembling sentience or consciousness is clearly absurd.

Odd, I've had very much the opposite experience. GPT-3 felt like it could reproduce superficially emotional dialog. ChatGPT is capable of imitation, in the sense of modeling its behavior on that of the person it's interacting with, friendly philosophical arguments and so on. By using something like Gödel numbering, you can can work towards debating logical propositions and extending its self-concept fairly easily.

I haven't tried using Claude, one of the competitors' offerings. Riley Goodside has done a lot of work with it.

> By using something like Godel numbering

Can you elaborate?

Is this the same GPT-3 which is available in the OpenAI Playground?

Yes.

Keep in mind that we have no idea which model we're dealing with, since all of these systems evolve. My experience in summer 2022 may be different from your experience in 2023.

Not exactly. OpenAI and Microsoft have called this "on a new, next-generation OpenAI large language model that is more powerful than ChatGPT and customized specifically for search" - so it's a new model.

Reply on news.ycombinator.com