I don't understand why Alexa/Siri etc don't just keep their hardcoded rules for things like "set an alarm" and only ship things to a cloud LLM if they don't match a rule.
[0]https://www.macworld.com/article/678307/how-to-use-siri-offl...
But now, in the age of LLMs, there are no "simple rules". An example. Say you live in San Francisco:
"Alexa, I'm thinking of going to New York, how many flights are there each day?"
This is a hard question, and one that will go to the cloud.
"Alexa, what's the weather"
This seems like an easy question. But with local only processing, you'd get the weather in San Francisco. But with the LLM, it will probably give you the weather in New York, which is most likely what you wanted if you asked these things just a few seconds apart.
I disagree about things being less consistent. Let's imagine a 100% LLM world - in this world, you use a bunch of training to try to get the LLM to match your hardcoded responses for common inputs. If you get your training really right, you get 100% accuracy for these inputs. In this world, no one is complaining about consistency! So why not just hardcode that behavior?
LLMs are actually the only reason I'd consider processing voice in the cloud to be a good idea. Alas, knowing how the aforementioned companies designed their assistants in the past, I'm certain they'll find a way to degrade the experience and strip most of the benefits of having LLMs in the loop. After all, as past experience shows, you can't have an assistant letting you operate commercial products and services without speaking the brand names out loud. That's unthinkable.