Hacker News Reader

Comment by MobiusHorizons | Hacker News Reader

why would you use an LLM for that? Seems like there are much better options available.

It's actually pretty hard to design a non-llm system that can detect all the possible variations:

Lights on. Brighter please. Turn on the light. Is there light in here? Turn the light on. Table lamp: on. Does the desk lamp work? It's a bit dim here, anything you can do? More light please. Put the lights on for the next 5 mins. Turn the light on when I come home. Turn all the lights off together. Switch the lights off whenever its daytime or quiet at home unless I say otherwise. etc.

If you don't support every possible way of saying a command, then users will get frustrated because they effectively have to go and learn the magic incantation of words for every possible action, which is very user-unfriendly.

I suspect ModernBERT can also be very helpful with these sorts of tasks, if you decompose them into an intent classification step and a named entity recognition step.

that entity extraction is where it actually gets really really difficult, even for LLMs since people will use 10 different names for the same thing and you'll have to know them ahead of time to handle them all properly. For either BERT based or llm based there's a bit of a need for the system to try to correct and learn those new names unless you require users to put them all in ahead of time. That said i've seen LLMs handle this a lot better with a list of aliases in the prompt for each room and then type of device when playing with home assistant + llm.

It isn't unreasonable to imagine one recognizable intent to be teaching new terminology. That would allow dialogs where the machine doesn't understand the object of the command and the human then says something like "when I say x, I mean the y" and the computer updates the training set for the named entity recognizer and does a quick fine-tuning pass.

Your examples include complex instructions and questions, but for simple ON/OFF commands you can go far by pulling key words and ignoring sentence structure. For example, pick out "on" "off" and "light" will work for "turn the light on", "turn off the light", "light on", "I want the light on", etc... Adding modifiers like "kitchen" or "all" can help specify which lights (your "Table lamp: on" example), regardless of how they're used. I'm not saying this a great solution, but it covers pretty much all the basic variations for simple commands and can run on anything.

Original Siri used a primitive sentence embedding and then nearest-neighbor of the vector to supported commands.

That is certainly a hard problem, but how do you know such a system is better than simpler command based ones? Systems like you describe have a much higher chance of taking the wrong action. Even humans do, when given ambiguous instructions. I know everyone loves to hate on Siri because it doesn’t do a good job of understanding anything complex, but I have always found it to be very reliable when you find the right command. As a result I use it much more often than google assistant (was team android/ pixel until 2020). I especially use it for timers, reminders, and navigation, and if my hands are not free, texts. Taking the wrong action breeds distrust, which I also think is not user friendly.

Reply on news.ycombinator.com