Friendship Is Optimal Is Science Fiction (Re AI)

> Nothing we have actually looks like this story! Nothing! [...] So what do these models look like? GPT style models predict the next word based off what it’s seen previously at training time. [...] There’s no machinery here that’s going to generate a bunch of plans and compare them against each other.

Yep. Sometimes I come across an explanation or metaphor which I just can't put down since it just has so much explanatory power, and for modern "AI" (read: LLMs) it's this: We've made what is independently a really cool document-generator, and we're dramatically over-hyping it to fool ourselves with collaborate fiction writing.

A word-appender machine that slaps together interesting half-recycled stories about a fictional $X doesn't mean the machine is an $X, even when $X = "a very smart machine."

> another AI which was shown a bunch of smiling people and made the dumbest, most limited inference about what we want.

That reminds me of the side-character Morpheus from Deus Ex (2000): The prototype spy/monitoring AI believed humans enjoyed being surveilled, since all of the elite bigwigs invited for a private demo would start smiling as it showed off all the information about them it was able to collect.

> "I was a prototype for Echelon IV. My instructions are to amuse visitors with information about themselves." [...] "Human beings feel pleasure when they are watched. I have recorded their smiles as I tell them who they are.