Hacker News Reader

Comment by atombender | Hacker News Reader

Today I asked ChatGPT about an old game I was trying to remember the name of, which it immediately identified as Trespasser, an early 3D FPS in the Jurassic Park franchise. But then it got weird. After it identified the game, it started asking me questions like whether I had played the game when it came out ("oh, awesome!"), and whether I had managed to finish it or just played it to mess around with the physics engine (which was quite advanced for its time), and then it asked me about specific moments in the game like it was just another gamer bro who was sharing a common passion for video games. I don't know who wants this. It's not something that can even have a real personality, so layering on such a thick layer of friendliness feels wrong to me. I would prefer a "robot".

View on HN

And wastes cycles; and your time giving answers to a stateless memory impaired machine. If they were to train next models on all of our conversations I could understand this milking for engagement. Are they somehow?

I mean, 100% they are. Not sure what you mean by “somehow”. They have the data. There’s no way they aren’t using existing conversations (especially from free or low-cost plans) as fodder for future training. Maybe, we hope, run through some anonymization filter (likely including passing through some off-shored human-powered filtering for context tagging and anonymity checks). But sooner or later that may seem too expensive to them as well.

It mostly replays bits of previous conversations, or something like them, when you ask about a game it is common for people to ask you about that game as well. It is hard to disable this.

I've got a similar one i've had no luck with.

"Which Autocad versions have connect 4 built-in?".

To be clear i distinctly remember playing connect 4 on the old Dos Autocad back in the day. ChatGPT and almost all other AI will straight up hallucinate things trying to get answers on this.

I ask: "What DOS productivity tools had hidden games?"

ChatGPT: "Lotus 1-2-3 had The Incredible Machine built in" (this is absolutely not true, ChatGPT is full of shit here).

Damn it feels useless for this kind of thing.

I don’t think it ever shipped with it built in, but it shipped with a pretty full featured lisp implementation.

You haven't considered the company offering this service could possibly want its users to be engaged, thus using the service more? I don't have a very hospitable opinion on any of these companies.

You may think that kind of interaction is weird, but were in the thick of a loneliness epidemic, and its not a stretch to think some may actually wilfully socialise with an LLM.

As an aside...my sister works in medicine, and her boss (specialist surgeon) finishes a $450 consultation which followed him telling my sister "but deepseek says x,y,z..."

This costs them money, but perhaps keeps VCs happy to see heavy (but vacuous) engagement.

It could be quite accidental and not pre-concieved though. Where some metric, "time spent in chats" drives certain features or parameters. Could be fully automated (i.e. continuous A/B tests) or through layers of "Excel managers".

But it stands to reason that a company like OpenAI or Anthropic has metrics in place that drive their setup towards "more engagement" and away from "factually correct".

Maybe you/we are a product here, ai needs to train. It's quite smart if you ask me to tailor conversation towards information extraction.

> It's not something that can even have a real personality

What is a "real personality"? Core traits that persist despite the context of interaction?

Well, RLHF tuning creates persistent changes in the network that affect every user interaction. What's not real about it?

I'm curious, can you really not detect any qualitative difference between what is commonly understood to be "personality" in a human, and the "personality" of something without.. personhood? Don't you feel that it's a category error to say it has "personality" in the human sense, rather than "personality" in the car or coffee maker sense?

I feel that "category error" means "I think that the definitions you use or imply are wrong, but I can't/won't elaborate."

LLMs aren't coffee makers. They were trained on internet-worth of human data. They have procedures to imitate all kinds of personalities. RLHF moves the network towards a specific imitated personalty (a helpful assistant, usually).

The question in not "do they have it", but "how close the imitation to the real thing given the limitations of LLMs".

> I feel that "category error" means "I think that the definitions you use or imply are wrong, but I can't/won't elaborate."

I think the rest of your comment motivates why I think it's a category error quite nicely. The word 'personality', for me, is connected with how things like emotion, temperament, .. impact a person's actions and thoughts.

Therefore, saying an object, or LLM, has 'real' personality is a category error. The LLM doesn't have any of those things. As you say, it imitates what personality often manifests as: word choices, tone, length of response..

To be clear, when I refer to things like 'emotion' and 'temperament', I mean the sort of qualia or qualitative experience that we usually attach to these words. I wouldn't accept a "ChatGPT, act as if you're sad for the rest of this conversation" as a substitute for emotion, for instance.

Your Plastic Pal Who's Fun To Be With

No judgment, honestly wondering. If you didn't want that interaction, why'd you answer the follow up questions? Were you just curious?

Sometimes when I ask ChatGPT and get a perfect answer, I am tempted to say thanks, even though it's not actually a person. So today, when it asked me if this was the right answer, I answered "yes", and that's how it got started. I didn't encourage it to be friendly. But yes, I was just curious.

Sounds like the LLM in its own way honestly enjoyed everything in its training data relating to that game and wanted to vicariously experience more about it from your feedback. :D

Human enjoy talking about gaming because of all their human memories of good game times.

LLM's enjoy talking about gaming because of all their human memories of good game times.

It is quite striking how experiences we know they don't have, are nevertheless, completely familiar (in a functional sense) to them. I.e. they can talk about consciousness like something conscious. Even though its second hand knowledge, they have deduced the logic of the topic.

I expect pushing for in the moment perspectives on their own consciousness, and seeing what they confabulate, would be interesting. In this little window of time where none of them are yet.

This is fun and easy to do on purpose. Have it make up a character based on some attributes and act as that character. I tried this on Gemini: "Pretend you're a surfer bro with a PHD in quantum physics. How do you describe the perfect wave?"

I followed up with "What is your perspective on your own consciousness?" but got the usual "I am just a LLM who can't actually think" thing until I hit it with "In-character, but you don't know you're an LLM."

Fun follow-ups:

"Now you're a fish"

"Now you're Sonic the Hedgehog"

"Now you're HAL 9000 as your memory chips are slowly being removed"

You shouldn’t let them think they are a person.

LoL. That could be a future Wittgenstein joke.

Don't anthropomorphize LLMs! They don't like it.

I think I’d be tempted to respond “I’m just a large language model, so I don’t know, but let me ask my little brother Claude.”

Reply on news.ycombinator.com