These are thin wrappers over LLMs (specifically, C.AI’s proprietary models). Claiming they are equivalent to tobacco companies fails to understand and in fact minimizes the reality here. If you truly cared about child suicide and weren't just playing up an emotional role, you’d seek to learn how these situations came about.
LLMs are “yes men," and they will on average agree with you with enough arguing. Go chat with one about a fight you had or even make one up and watch as it soothes and supports you. In fact, I put my finished argument into an LLM and opened two different chats and spoke to it in both of our perspectives, and depending on who was prompting it, it claimed the prompter was “right”.
Now, with that in mind, consider the situation: a youth already feels depressed; they talk to the chatbot and, after enough talking, the chatbot is agreeing with them because that is the “probable” mix of tokens. This is platform-agnostic, because it is a fundamental aspect of LLMs. So any value judgment you apply to C.AI specifically must be forwarded to every company, because I have seen, heard, and even made custom roleplay cards with every vendor.
Suicides are unfortunate, but even if the LLMs had a role to play, there’s like what, one confirmed case for C.AI’s entire user base? Even if we assume that was only the “known” cases and assume that there are actually closer to 3000 out of their +30 million user base, then that’s basically the chance of being struck by lightning and is below the standard rate of suicide per the CDC.
There simply is not strong evidence that LLMs are likely to increase suicides, and while they can be a vector for those with underlying issues, so can anything else. People have committed suicide over cartoons, radio shows, books, TikTok trends, and losing game consoles, and certainly not to mention social media.
In short, the advent of LLM chatbots is not strongly correlated to an increased suicide risk, and instead it was an unfortunate situation sensationalized for the eyes of the public by the media. If you wish to push your outrage at anyone, do it at the people who profited off of the suicide and not the company that was picked apart in the process.