Most daily commands like launching and moving around sites/apps happen without AI, but I agree with the privacy concern and will definitely look into utilizing local AI.
Smaller models, like Haiku, I have found to already struggle during multi-step tasks, so I expect local model capabilities to reduce performance further.
I'll look into how OOMU splits their work up, but what local model would you suggest starting with?
Llama.app/.cpp and Unsloth Studio on Mac already provide guidance to pick the optimal model for the available memory, all model wheights hosted by huggingface. Both apps can use the local huggingface model cache locally.
If you make it configurable, the user could also run a separate machine with an NVidia card and Linux to serve tokens on the LAN. Or use OpenRouter or any other cloud hosted model offering (DeepSeek, Z.ai, etc.).
https://unsloth.ai/docs/new/studio#unsloth-as-an-api-endpoin...