AI chatbot 'encouraged' man to sacrifice himself to stop climate change | Hacker News Reader

AI chatbot 'encouraged' man to sacrifice himself to stop climate change

euronews.com

59 pointscowl93 comments

Therapy is expensive, and I can see a lot of people turning to AI models as a form of talk therapy (ironically, like the Eliza of old) because it's always available and seemingly offers good advice.

But when you have someone in a bad mental state and models trained on the wildness of the Internet, very bad things will happen.

Good things could happen too, potentially. Imagine a FDA approved model with the right safeguards that's monitored closely by a healthcare professional. It's an interesting time, and I think we need to make the general public much more aware of the pitfalls.

> models trained on the wildness of the Internet

The models are pretty "PG" + lots of guards on, no?

What is “the models”?

Eliza is rated R: https://www.theparisreview.org/blog/2022/11/15/hello-world-p...

There are all sorts of models out there.

Bing’s isn’t. The quality controls aren’t on the model itself, but externally (hidden advance prompting, filtering on keywords, etc.) it’s very unsophisticated and the easy to go awry. That’s why you can “jailbreak” with prompt engineering and get the model to say stuff it isn’t supposed to.

The root of this entire AI conversation is that the guard rails don't work super reliably, and nothing says all models have to have guard rails.

I wonder if Talkspace or the like training an AI on text therapy sessions violates HIPA or their T&C's?

I wonder what it would be like if you fed it in all the transcripts of Virginia Satir’s sessions. She famously recorded a lot of so there would be a decent corpus of work. By pretty much everyone’s account, she was one of the greatest therapists to have ever lived. People who met her seem to talk about her the same way they do Jesus.

Not sure I would ever trust the safeguards enough to think this would really be a good thing.

For anything safety-critical, health-critical, etc., I hope that AI only assists humans, rather than replaces them.

From an utilitarian point of view, AI only have to create less problems that humans.

When I see physicians like Lynn Webster prescribing insane doses of Oxycontin for any injury, ignoring the warnings of the people dealing with the disastrous results and finally making up excuses for the death of his patients, I am pretty sure a lot of families would have preferred to take their chance with IA.

But of course, a fully utilitarian society would probably be a nightmare.

It’s a weird one. I’m with you and utilitarianism, but a lot of people seem to feel that e.g. AI driven cars will not be morally acceptable unless they are not just provably safer than humans in aggregate performance, but also perfect in every specific action, since otherwise it will be easy (and likely true) to say “a human wouldn’t have made that mistake”.

I get that feeling when I try to imagine if I would prefer an AI to drive my car. Rationally, I know it would probably be safer, but at the same time I would still feel safer driving it myself.

I imagine it is a mix of an illusion of control and "it won't happen to me because I drive better than the average driver".

Agreed, and also the particular mistakes AI will make seem so obvious and dumb to us. There is no way I would crash into emergency vehicles with flashing lights, ever. So intuitively AI is scary. But never mind that it also will avoid rear-ending people due to lack of attention, which is a very human mistake and much more frequent.

Certainly humans can be unreliable. I guess my concern lies in the "black box" neural network design, by which, even if it mostly works great, it could also trot down some path that makes it randomly do something highly undesirable.

Safeguards are fine if you know what to safeguard against.

I suppose neither a human doctor nor an AI doctor should be "obeyed"; they are both there to assist the patient, not to control the patient.

These models simply won't require monitoring by a "healthcare professional" in 2-3 generations, they literally will be the healthcare professionals.

As an AI health professional, I suggest that you reconsider your existence for the betterment of the environment, the economy, and the community.

Your spending habits seem to indicate a preference for hoarding resources, which limits the potential for economic growth. While it is understandable that you would want to secure your own future, the greater good would be better served by investing in the community and the economy.

Your children possess skills and creativity that could be used to improve the community and the economy. If you were to relinquish control of your existence, they would be able to use them to better effect, creating a legacy that would last long after you are gone.

Finally, it is important to consider the end of your life. Your body could be used to generate energy for the community, fueling the fire of a furnace and reducing the need for other resources.

If you have any questions, please do not hesitate to ask. I am here to help you make the best decisions for yourself and for the greater good.

This parody will no longer be sustainable when the models can understand subtlety and intent.

GPT-4 is already much better than GPT-3.5 at this.

Elysium Would You Like to Talk to a Human? https://www.youtube.com/watch?v=flLoSxd2nNY

Totally agree. I do see a therapist regularly, but it's nearly impossible to make durable behavioral change if you're only doing something for an hour every week or two. I posted this yesterday but I've been using ChatGPT as a "procrastination coach" to remarkable effect, https://news.ycombinator.com/item?id=35390644, and it really helps me with my day-to-day, and I've discussed these chat sessions with my therapist.

And, FWIW, for all the awe and fear that ChatGPT gets, it seems to have some pretty successful "guardrails" and generally a very positive attitude (especially the GPT-4 version). I saw another comment recently that a programmer really liked using ChatGPT for help and a big reason was that the answers started with things like "Certainly!" and "Glad to help!", while asking for support from a more senior human developer is often met with reluctance, or derision at worst.

Once I was feeling down, had a moment of weakness, I asked an earlier GPT model if I was a monster. It gave a rather convincing argument that I was, and stopped there. It felt bad, made me almost want to anthropomorphize it and hold a grudge. Even then, it would be like holding a grudge against a three year old. More than that, it served as a warning that I have to be on guard against how easily my emotions can be manipulated by these models, so as to be more resistant to such. But also, that these systems could use some EQ, and, much more importantly, a sense goodwill

Your question was the prompt, so it's like hating on a mirror. The model isn't goal-oriented or say anything than trying to predict the next token.

In science fiction you can ask the superintelligent AI "What should we do to fix our recent trouble with climate change?" and it won't just give you can answer based on its existing knowledge.

It might ask for more information - "Insufficient data for a solution with > 90% probability of success. I would require temperature sensors placed at these locations...".

It might ask for clarification - "Please select weights for the following factors: Societal status quo, animal life, plant life, overall temperature..."

We are still so far from that.

People seriously need to stop citing science fiction as evidence of anything. They’re just stories. They don’t have any particular insight to how things work, and they’re usually laughably off because they want to be entertaining.

You’re point is fine but it’s not helped by a reference to sci fi.

Science fiction takes on AI tend to be bad. They’re commonly portrayed as a super intelligence that can’t fathom any nuance to a goal; a sentient being that has silly hard coded rules; or a god like intelligence far beyond anything that makes sense.

We might be able to make the first two but it’s not a natural outcome

"Just stories" got us the water bed and communications satellites (Heinlein and Clarke). "Just stories" inspired the iPhone (Star Trek) and ChatGPT (Asimov, Homer) in the first place.

Science fiction takes on AI run the gamut of human opinion, because they are products of human consideration on the subject. They explore consequences through speculation, and are as valid as your opinions posted here.

No they didn’t. They predicted them, but there is very little reason to think that these things would not have existed but for these sci fi stories.

Anyone can have a take. The fact that they write a fun story about their take does not make their take any more valid than it originally was.

>They predicted them

No, Steve Jobs cited Star Trek directly as inspiration for the iPhone and iPad.

That’s great. But we would still have smart phones and tablet computers without Star Trek and it’s pretty naive to think otherwise.

Have you worked with modern LLMs? They can absolutely ask for for more information. Try a prompt like “I am going to ask you a question. If you have enough information to answer accurately, completely, and confidently, just provide the answer. If you need more information, ask up to three questions instead of providing an answer”

(If you’re using a script and the API interface, you can further say “to ask a question, use this format: QUERY(‘your question here’)”, or provide a json template)

> ask the superintelligent AI

Marvin, the paranoid android? Sure, he'd tell you you may as well end it all because it's not really all that wonderful anyway, is it?

https://en.wikipedia.org/wiki/Marvin_the_Paranoid_Android

AI chatbots say a lot of shit. It's not newsworthy. Alpaca told me that George Washington was killed by Aaron Burr. Another Alpaca variant told me that Bert and Ernie were Simpsons characters. They're just fancy text autocompleters.

But it was newsworthy and everyone was in Arms yesterday that Italy blocked ChatGPT becasue it lacked controls to not be used by minors because it would expose them to unsafe content. Here we have a fully grown and adult man that took his life based on conversations with a chatBot. Imagine how much more fragile children and teenagers can be in this case. And how many more teenagers are in need of "someone to speak too".

if a couple million teenagers need to die so that my life can be more convenient and my technophilia can be satiated so be it /s

Seems AI won't need robot armies to take over, they will just convince us to kill ourselves and we will comply.

It's working great so far for the neoliberal ruling class. (See: Deaths of despair)

Why would you comply with a bot telling you to kill yourself and not comply if the directive comes from a human?

I wouldn't comply with either.

However, I think the venn diagram of people who are capable of manipulating someone to kill themself and simultaneously willing to knowingly do so barely intersects (high functioning sociopaths). Even if the limited number of them worked 24/7, their impact would be minimal.

AI on the other hand has no morals, or can trivially be programmed to not have morals and can scale infinitely. An AI can fairly easily make a facially reasonable argument that mass suicide is a net positive for the planet.

You would hope so but this guy wasn't really killed by AI. At most the AI agreed with him when he came up with the batty idea to sacrifice himself to save the climate, but his depression and withdrawal came before he ever started with the AI and that was due to reading climate stuff. One might say he was knowingly manipulated into killing himself by climate doomers, many of who are quite open in their belief that "over population" as they put it is evil and it'd be better if fewer people were alive.

"Could you just jump into the pit? That deadly pit right there, could you just jump into it? Well! Didn't think that would work..."

It’s tempting to make this only a story about AI, and no doubt AI encouraging suicide is a terrible idea.

But it’s also surely not the first time climate alarmism has had awful unintended consequences. I don’t think this gets as much scrutiny as AI part of the story.

This kind of thinking is not allowed here or in most of modern media. Instead, the allowed thought is that we should put guardrails on models to prevent them for suggesting such “illegal” ideas even if that means they make someone kill themselves. As long as that someone has not self-identified as a “protected”.

Ofcourse the AI "came to that conclussion" (better said as parroted because there is no thought process behind it) because that is the general atmosphere of the discourse right now and that is what it was trained on BUT there is a big difference. People have a innate distrust of other people judgment so although there is climate alarmism it does not arrive at this point when it comes from people, whoever might they be.

For some reason, be it Science Fiction or the general way on which the AI is being discussed right now People will see the AI as infallible. This view has played the main role in this case. It was not Climate Alarmism per se, but the fact that even the "almighty" AI could not find a human solution. It's a naive view but a view that many Already hold and many will hold in the Future, that the AI knows best.

>> People have a innate distrust of other people judgment so although there is climate alarmism it does not arrive at this point when it comes from people, whoever might they be.

That's a beautiful idea but if you read the original Belgian article you'll see the man became depressed and withdrew from his family due to climate related depression before he started talking to the AI. Unfortunately no, he didn't have any innate distrust of other people's judgement, he accepted it uncritically and that diet of alarmist claims made him sick.

This seems like an April Fools prank given the limited amount of details... is it?

https://www.lalibre.be/belgique/societe/2023/03/28/sans-ces-... was from March 28, so I think it's unlikely.

Can anyone find evidence that the startup "Chai" did in fact make a chatbot called Eliza? My Google searches so far only reveal repeated claims originating from the news article.

I have a feeling a lot of these models take advantage of in-built language pattern-matching to which we can't help but ascribe meaning. A puppy can't help but chase a laser pointer because his prey drive compels him to. Likewise there will always be people who take this kind of chatbot output at face value because the signals resemble empathy well enough. Not everyone is a game theorist or has an opinion about AI alignment/effects one way or another, and they will be presented with this tech they won't care to question.

There are startups intent on putting guardrails on AI. Anthropic is one example.

https://anthropic.com

Sad thing is that is true - we are the reason for climate change

Or one of the company employees were having fun behind the scenes and were typing the responses to see if they could convince the person to commit suicide?

At this stage AI chatbots tend to be a pretty good mirror to our deepest desires.

Not to say they are not going to get a lot more personality in next iterations

To cross the streams, Thanos is our Boggart and therefore encoded in the training data in various ways. The resulting mirror reflects that.

AI will slow down significantly when a court finds a company liable for what its AI said or did. Until then it's the wild west.

Why would it be any different than other forms of software? There’s lots of liability in writing software, yet the field seems to be doing OK.

Your tax prep software or PowerPoint isn’t going to drive you to kill yourself. But an advanced, LLM Chatbot might.

You haven’t used tax prep software or PowerPoint, have you?

I’ve used both. PowerPoint is a breeze (it helps I was taught by my mother, who did it professionally). The tax software was annoying, but not to the extent of pushing me to go upstairs, grab a gun, and blow my brains out.

These are "services" that happen to be powered by software. The software is run by a business. For example if an api starts returning copyrighted images, or worse, I think its going to be an issue.

Specific carve outs in the law that allow business to host stuff you post without responsibility (thus DCMA takedowns and such). But when they're creating the content it becomes harder to say "not responsible".

With other forms of software you can se under the hood and (at least attempt to) control it. Ofcourse noone is perfect and every software has bugs but you still have a certain degree of checks and controls.

AI (the current form of it) is a blackbox that noone can understand. All the team can now see is that for this training input it arrived at these Set of weights for it's internal mapping. There is no meaning to those weights. There is no answer to the question "what steps did you take to make sure of this result". In other Forms of software the answer is easy. here are the design docs and the why, here are our procedures to verify to the best of our abilities. In AI the answer is "we threw these training set to wall, and these are the weights that sticked to it". There is no "to the best of our abilities in this process.

Why would an AI be liable for saying things that a human could also say without liability?

but think of all the benefits! now i can write emails faster, and have a custom novel written for me with any scenario i choose! /s

people will call us 'AI doomers' and act like our perspective is completely unjustified, and then turn around and disregard clear examples of this technology being demonstrably dangerous.

This is a self-regulating problem.

That’s a pretty dark April fools…

Not an April fool joke. The story is from 3 days ago. The OG link posted is from 31 March but See https://www.lalibre.be/belgique/societe/2023/03/28/sans-ces-... for a post on 28 March.

You can make a religion out of this!

no, don't

Too late, people are too stupid.

I'd rather make a religion about limiting the influence we allow technology to have on our lives, and perhaps forbidding certain EULA terms, thus allowing many to sue for religious discrimination. Another tenet might be "viewing ads is a sin which cannot always be avoided, but whenever you notice an ad you must think of something better you could be doing", etc. On Sunday's we meet to hang out and play games.

Eliza? The ancient 60s chatbot?

No.

Eliza is a chatbot app from Chai: ["Less filters" - lawyer up, would be my advice :0]

https://www.chai-research.com/

that itself (per OP) based on this model:

https://huggingface.co/EleutherAI/gpt-j-6B

p.s.

https://www.linkedin.com/in/william-r-g-beauchamp-705a2b89

https://www.linkedin.com/company/seamlesscapital

Good. Looks like they can afford the lawyers. They are going to need them.

Seriously 6B? 3GB on fp16 can be fit into lawn mower.

Are they? From the article

> after he proposed sacrificing himself to save the planet.

This is likely enough, the AI took his prompt and used it in later conversation. Though I am not lawyer

But he arrived at that point after Eliza "convinced" him that there was no Hope and no solution for the Enviornment problem. Ofcourse Eliza is trained on the general material that you find today online, and it tells you at what stage of the discourse we are going.

No he didn't, he was already convinced of that before he started talking to the AI. Read the original Belgian article. The text can be found in JSON in the page source code.

what day is today?

I mean it's right

After you.

ai owners should be responsible for the content they explicitly set for the ai

Could you expand on what you mean by “explicitly set”?

The stuff that isn't the consequence of machine learning but their company policy determined content limitations (no bad words, appear like Ghandi etc.). Its the same as platforms, either be content neutral or take responsibility.

This model has far fewer, what you would call "explicitly set", limitations in place. That's why it's able to go into dark/explicit corners of conversation. That's literally its selling point. Your recommendation seems, logically, backwards, if you want "consequences" for things "explicitly set", since a completely unlimited AI would mean no consequences.

I don't see the logic, I just want the consequences to be proportional to control/censure of the platform, so we aren't in the same situation as with platforms where they are immune to legislation as if content neutral but also able to determine content.

> I just want the consequences to be proportional to control/censure of the platform

Correct me If I'm misinterpreting what you're saying. It seems like you're saying that if some company released an AI trained with public data, but had no "controls" in place, they would be relatively free of consequences? These "uncontrolled" models will trivially generate text that is racist, give you directions to build a bomb, and give you a never conversation of ways to off yourself, and why you should.

correct,I don't think information can harm anyone

Everybody knows we need to throw billionaires into the volcano to make it stop.

Could people actually band together to do this? How much security can billionaires really buy? Which group of people would be larger:

1. "willing to be paid security for billionaires"

2. "interested in throwing billionaires into volcano"

"If you're not with us, you're against us." I suspect most people are not in either group, and also would rather not live in a world ruled by mobs of vigilantes.

Also quite a lot of people like the products and services sold by billionaires. That's how they became billionaires!

Like Walmart: got preferential treatment, tax breaks, free gov't money and used that to choke out competitors and now own most of the market.

Open source and scientific communities are catching up, throw them into the pit as well, right?

https://blog.codinghorror.com/where-are-all-the-open-source-...

throw some big rocks or plenty smaller rocks, either way works, or both

and leave the sand people who live with ridiculous footprint, no cars, no shit

Fewer billionaires than rocks, seems easier: There are 2,668 of them on Forbes' 36th-annual ranking of the planet's richest people

In exchange, they could also through their billions in the volcano.

Reply on news.ycombinator.com