Hacker News Reader

Comment by lolinder | Hacker News Reader

> The chance that chatgpt just hears your example and than does the right thing is nearly a no brainer.

ChatGPT can't do anything. Someone has to, somehow, wire it up to actual physical or digital controls. Intent recognition and slot filling still have to happen somewhere, and that means defining discrete intents and slots.

GPT may be better at mapping from speech to intents, but it can't magically interface with APIs that haven't been defined.

EDIT: To elaborate a bit, the problem in OP's story isn't caused by a bad language model, it's caused by no one at Apple thinking to define different "alert volume" and "media volume" intents. Current language models are plenty good enough to recognize the distinction, so simply adding ChatGPT won't be enough to make any feature work unless someone at Apple predicts the need and writes the interface.

View on HN

If I tell my computer to "froblate" it says "command not found" not "command shutting down".

If I tell it to "shtudown", it said "did you mean 'shutdown'?

There is an AI failure in distinguish "mispronounced command I know" from "command I don't know".

I'm not convinced that ChatGPT would do better than existing models in that regard. Using it for intent recognition is already a bit of a square peg round hole situation.

Basic intent recognition models are trained to produce a single neuron per intent as the output, which makes it pretty easy to use the activation levels of the output to decide whether to perform an action, confirm an action, or ask for clarification. You just need to check if the certainty is below a certain threshold.

With ChatGPT you'd have to encode the intents as text of some kind (JSON?) and hope that it doesn't just hallucinate an intent that your APIs don't have when it's faced with ambiguous input. You could probably have hallucinated intents map to a decent-sounding error message, but that feels more brittle to me than the existing approaches.

It can interface with apis it only read about.

Ask it to write an SQL query and it will.

Give it upfront context and it will be able to produce API commands.

The voice recognition can trigger chatgpt API 'create API command for the following text's.

The practical problem is cost and hardware.

But this is closer than we ever were.

Having ChatGPT write code that you immediately turn around and execute is a very risky proposition for all kinds of reasons, the least of which are crashing at runtime or failing to compile. If you're having it produce code in a Turing-complete language, you've opened yourself up to the worst kind of prompt injection attacks.

A much safer approach would be to have it produce a JSON object that is interpreted by the wrapping code. This allows you to inspect the interpreting code and provide guarantees about what can and cannot be done by users.

But neither execution model solves my main point: someone still has to write the interpreter (or the library functions if you go the dangerous route). Someone has to think ahead and guess what the user is going to try to do and provide APIs that allow that. And that work is most of the work: our existing intent recognition models are actually not bad at all at their job, the failings these voice assistants have are almost all to do with predicting the ways in which users will try to interact with the system.

Have you tried it out?

Tried what out? Code generation and execution? Or building a full-featured voice assistant?

The code generation and execution is trivially problematic. I don't need to try it out to know that if someone can coerce one of these bots to spit out its prompt, someone could coerce it to execute dangerous code if it's wired up to an interpreter.

As for the other part of my analysis, in order to empirically test it I would have to build a full-featured voice AI, release it to millions of users, and see if I was able to predict everything they tried to do. Given that my prediction is that even Apple can't do that well, I'm not sure why you think I would bother to try it out.

If an engineer told you that a bridge would collapse if built a certain way, you wouldn't insist on trying to build it anyway just to be sure. Most of the time in engineering you can't run a full-scale test, you have to make do with analyses. If you want to critique my analysis, I'm all ears, but "you haven't actually tested it" is not a critique.

I meant chatgpt

Yes. I've used it quite a bit. It's impressive, but the hype is excessive.

All of my analysis is based on my usage of ChatGPT and my understanding of the underlying model. The hype is largely driven by people who don't understand how it actually works and think they're interacting with an artificial general intelligence.

It's a very impressive language model with a lot of applications, but with many fewer applications than the hype would suggest.

I'm hyped not because I think it's an agi.

I'm hyped because I threw normal text (sentences) against it and it always did what I wanted and it didn't matter if it was bad English or bad German.

I didn't even try promt hacking or promt tuning because natural language already worked so good.

It's a tremendous good 'normal human' text interface.

Reply on news.ycombinator.com