What can/does the Rabbit add to it above what a simple string concatenation would?
Weather was just an example. You can't predict every combination of weather related questions that someone might want to ask and force those into predetermined response strings.
The idea is to be able to ask things like "What days this week are best for having an afternoon picnic?" and get a reasonable answer back.
If your goal is to hide as much of your personal information as possible from 3rd parties, you're not in the target audience for these products. Nothing wrong with that, it's just targeted at a different demographic of people.
The llm is there to understand that you've said "what's the weather?" not, say "call me an uber", and then to collate the responses from potentially several calls to websites and produce a natural language response.
There is a layer of upsell/hustle/scam[1] on top of that where they said they had a "large action model" which learned your preferences and understood how all these websites worked sematically so the actions would be robust etc and a bunch of other stuff which turned out not to be true.
[1] depending on your point of view. I would encourage people to watch https://www.youtube.com/watch?v=NPOHf20slZg and https://www.youtube.com/watch?v=zLvFc_24vSM and do whatever personal research and make their own minds up
Typically the LLM queries external tools as appropriate and then incorporates the result into the response.