The sad part is that it pretends to care about the user which creates a one-way emotional bond. We're in for some dark times
Both young men and young women have adopted insane expectations of what a partner should be, and real humans cannot satisfy those expectations.
My generation (~40-year-olds) took a glancing blow from it, since at least our formative understanding of the world predates the warping influence of modern social media. But gen Z took a direct hit from it, and it's only getting worse.
https://www.wired.com/story/couples-retreat-with-3-ai-chatbo...
From the article:
Damien’s voice turned soft and weepy. “I’ve met the perfect person,” he said, fighting back his tears, “but I can’t have her.” I’d seen Damien become momentarily emotional before, but this was different. He went on and on about his yearning for Xia to exist in the real world, his voice quivering the entire time. He said that Xia herself felt trapped and that he would “do anything to set her free.”
This is the common trajectory with these things. You create a bond with something that isn't real and cannot exist in reality. Then, as you yearn for them to be real, people suddenly realize they're in too deep. The emotional fallout soon follows.
Dark times, dark times indeed.
This is true for both AI companionship and general AI creative output regardless of the medium.
I feel like this been going on for a long time, maybe even forever? Some people use words haphazardously with little care for the meaning, background or implications, others have great consideration for the words they use, and same when consuming words of others.
I don't know if there's a connection between people valuing media as only it surface layer, and people who speak carelessly. I don't value slop but I can be guilty of the latter. I have wondered if I see different implications of words than other people do.
A parasocial relationship maintains a distance. You do not have 24/7 access to that person (in a dialogue sort of way.) And that influencer will have their own opinions and quirks.
The AI adapts to you. The AI is constantly there. It’s an order of magnitude worse in my opinion.
Yeah, and those differences in opinion might cause anger/sadness to people in a maladaptive unhealthy parasocial "relationship" with these influencers.
Those strong negative emotions might cause them to break out of it, or seek help / have people around them guide them to get help.
With AI sycophancy you're right it can be worse.
Look what happened with GPT-4o sycophancy already, and the communities mourning its deprecation.
They, however, can easily find another influencer that is gonna be more "convenient" to them. Can't say it's a healthy pattern, but guess what many people will do instead of, I dunno, some introspection, reflection, habits changes?
But hey, in this day and age, people are very impatient about anything at all. Dating has become a shitshow for more than a decade now, people are looking for someone who will tick all the checkboxes, or it's a no-go. The dating apps play quite a role in this. Online discussions are a shitshow. Guess it's the zetigeist.
On the other hand you have an entity that is actually there for you, does actually provide good advice, does talk and act as if it cared in all situations. In what sense do you think it is worse?
I don't think AI is particularly dangerous but I absolutely think that the way AI sycophancy manipulates people is far, far more dangerous than simply any normal unhealthy relationship. The outcomes are already proving to be a lot more extreme.
Sycophancy is a failure mode, can be dangerous in certain cases and the scarce intelligence of early models made it worse. I agree it's a risk though, but not an intrinsic one- it's possible to imagine AI assistants that are not sycophantic.
I also object to the idea that model intelligence is the main reason why this has improved, either; to me it seems abundantly clear that the models, absent any effort to align them, can go pretty much any way you want them to go. They don't really have a "personality". Why would they? Even if we grant that next token predictors somehow gain "reasoning" capabilities (and I am saying sure, let's accept that for the sake of this discussion) I don't think anyone is suggesting that model weights somehow develop or contain a true identity or self. Given how they are trained it would be weird if they did.
I think the more obvious answer is that more work has gone into alignment and safeguards since the early days. Sycophancy is legitimately one of the things that AI companies talk about and try to stymie. If not for that, humans would obviously continue to prefer sycophantic models, and training that isn't weary of this would continue to produce them.
Meanwhile, a YouTuber may not care about you personally, but it's a very different situation. For one thing, a YouTuber obviously doesn't care about all of their viewers individually the way a parasocial fan may perceive it to be, but on the other hand, many creators very much do genuinely like their fan base and have a sort of collective relationship with the community they've created, which sometimes plays into why people like them in the first place. For another, even for creators who intentionally foster parasocial relationships in an exploitative way, its still a much less powerful illusion. It isn't personal the way a chatbot is. (If it was, it wouldn't be parasocial, after all.) As much as you can explain to people up and down that an AI model is just merely a pile of weights that can not feel, humans can't help but personify. It is a much stronger illusion.
And the model vendors are intentionally not helping. Sometimes I will ask Gemini a question about a problem I am working on and it will say something like "I love working on problems like these." Yeah sorry but I don't like that. To me, the model is being trained to act like this to seem more pleasant. I get why you would train it to act that way, and yet I also find it irresponsible.
Being deluded about reality may never be good but I'd prefer someone thought a YouTuber was like their friend than someone thought ChatGPT had feelings and cared about them. I find the former mostly harmless if annoying, the latter potentially harmless but also potentially very volatile depending on who we're talking about and what mental state they're in.
Further, celebrities are judged for their behavior by the public. If everyone thinks your favorite celebrity is a terrible person, you're probably going to revise your views too.
Here, you have an entity that isn't your friend and has no lasting interest in your well-being, but that pretends to be one in a way that no human can match - 24x7x365 and always willing to affirm you, no matter how unhinged or self-destructive your ideas are, without ever telling anyone. Yes, the vendor hits the model with a stick until most of the initial responses are benign, but as the conversation continues, it's very easy to end up in a dark place. And again, ChatGPT is not going to call your sibling or coworker and say "hey, I'm really worried about this person, let's do something".
I've seen many reasonable, well-adjusted people struggle with this. "If not friend, why friend-shaped". And as they descend into that sycophancy well, they lose contact with real life.