Hacker News Reader

Comment by pixl97 | Hacker News Reader

We need a name for this, something like genie effect, but that doesn't sound creative enough.

But working of the fables of the genie, we have and all powerful wish machine where the user mistakes they are telling the genie with intention of their thoughts, whereas the genie is looking at all possible decodings of the words they are saying without knowing the users thoughts. If you do not verbalize exactly what you want (and have done some thinking about the ramifications of what you want) you might end up squished under a pile of money, or some other totally ridiculous situation.

Unless you share your entire life with the AI/genie you will always have to give more detail than you think. Hell, in relationships where people live together for years there is always the possibility for terrible misunderstandings when communication breaks down.

View on HN

The age-old bane of software requirements: "It's exactly what we asked for but not what we want."

I've used an analogy of magic and spell casting to prompting in the past, but I've since been convinced that it's risky to describe it like that because it implies that this stuff is all-powerful and impossible to understand: https://simonwillison.net/2022/Oct/5/spell-casting/

It seems to be a theme with AI.

Scale.ai (no affiliation, not a customer) has a product called Spellbook

https://scale.com/spellbook

Yeah, I've definitely had informal conversations with AI experts where they've talked about collecting spells for their spell books!

“Any sufficiently advanced technology is indistinguishable from magic” -- A.C.C

I'm wondering if we'll reach a point where it is impossible or nearly impossible, and what value of 'understand' we assign to it. For example deeply understanding x64 processor architecture. Does any one person understand it all, very unlikely at this point. The investment the average person would have to perform to accomplish even part of that would lead most of them to seek other more fruitful endeavors. Biology would be a good example of this. Nothing about biology is magical, and yet even simple systems contain such an information density that they are at the impossible to fully understand point, and that's before we start layering them into systems that have emergent behavior.

This was the theme of Bedazzled, the Peter Cook and Dudley Moore film (remade in 2000), itself a twist on the Faust legend.

In this case though, the increasingly desperate engineering of the prompt was each time intentionally side-stepped by the devil, though much like GPT, he couldn't help being like that, it was just his nature.

We need a name for this, something like genie effect

Investigating why the prompt isn't working:

"Debazzling"?

Reply on news.ycombinator.com