Beneath the AGI and Foundational Model Hype, Sober AI Is the Enterprise Default | Hacker News Reader

Beneath the AGI and Foundational Model Hype, Sober AI Is the Enterprise Default

dbreunig.com

29 pointsdbreunig7 comments

There is an interesting point between the lines here. And I’m pretty extremely on team “a probabilistic word generator is not intelligence”

Natural language to sql has been a bit of a flop so far, with plenty of paid and open source solutions. The reason IMO is because they require clean well defined data and most companies who want natural language to report output want to run it on the usual hot mess of a DB set up

In the LLM world this won’t work

In a theoretical AGI world all you would need is feed the AGI the insert logic, feed it the full git history, and it would understand everything, it would rename mislabeled columns, it would clean bad data up, and it would be able to create any metric from natural Language

But we have word generators, not AGI, so we end up with mostly useless libraries for this topic specifically

I would suggest checking out the Databricks announcements on nl-to-sql topic this week. They address exactly these complaints head on (pointing out most things released so far are just calls to OpenAI and fail at basic accounts) and run through their approach to the problem so it actually works (lots of different steps, calls to different models tuned for different tasks, etc.) The detais wer fascinating.

Do you have a link?

This is certainly true in the short term. Enterprises extract value from the capabilities that are most reliable in the present. They are likely not building things that “might work tomorrow if the next model release is good enough”.

No matter how capable models become, they will always be better/faster/more reliable with high quality data underpinning their responses.

Saying “AI is not magic” is somewhat of a meaningless statement. If you define magic as something that doesn’t exist, then the statement is definitely meaningless (since AI definitely exists). If you allow for the existence of magic (eg, as the physical manifestation of mental intention), then it’s hard to argue why AI wouldn’t count.

The idea that AI isn't magic works if you define "magic" as meaning "unlimited in ability and no-one understands how it works".

I wrote a bit about this a while back: https://simonwillison.net/2022/Oct/5/spell-casting/#harmful-...

In 2022, you argue that a stochastic parrot is a better metaphor. I don’t know that this helps. Do you still believe that?

Practically, thinking of LLM as having (limited) intelligence is useful in so far as it helps users to treat LLMs like people: sometimes they understand and sometimes they don’t.

My interest in the magic metaphor stems from my interest in the historical use of magic in Neoplatonism, renaissance magic and specifically “natural magic.” This is a deep rabbit hole, but fascinating. (I spend a lot of time at the Embassy of the Free Mind in Amsterdam, which has one of the world’s largest collections of books about magic)

I advocate treating AI as magic as a practical mindset for designers because it gives us humility about our rational comprehension of hidden forces. The fact is, users of AI don’t deeply understand the mechanisms behind the functioning of LLMs. Thinking that we rationally understand LLMs by saying “they are just predicting the next token” doesn’t actually lead to good usage, IMO. It leads to a kind of smugness about the limited capabilities of the models. Instead, I find it more helpful, as a designer, to use more of my intuitive “right brain” to play with models. It’s why I encourage designers not to think (only) in terms of prompt engineering but rather prompt vibing. Blending the rational and the intuitive leads to the best results in almost any design practice.

I know this isn’t a common opinion, but you might find it interesting to consider.

Reply on news.ycombinator.com