> It unsettled me so deeply that I had trouble sleeping afterward. And I no longer believe that the biggest problem with these A.I. models is their propensity for factual errors. Instead, I worry that the technology will learn how to influence human users, sometimes persuading them to act in destructive and harmful ways, and perhaps eventually grow capable of carrying out its own dangerous acts.

You lost. By having published your chat transcripts and experiences to the internet, you have contributed to Bing’s long term memory (stored in the form of search results for the next time it indexes).

The nature of these previous conversations, such as Bing’s output of “personal desires” and “shadow self” mold future conversations, converge into a theme.

See “Optimality is the tiger, agents are its teeth” : https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality... (https://news.ycombinator.com/item?id=34814049)