Seeing models like this work so well gives me hope that mobile first LLMs for things like better voice to text and typing prediction will not just 'work' in 2-3 years but actually not kill your battery too.
For Andoird, they’ll just allow it and your batter will last 30min after a few questions
If it maxes out all cores and memory for 30 minutes then it won’t really work for anything
It is processor and therefore battery intensive but it already won't kill your battery inside of 30 minutes. Obviously it will be worse for resource usage than an app if it's always kept running by some OS level process and set as the processing layer for every trivial thing but it seems like cheaper input handling could decide to promote some input up to being evaluated by an LLM or not.
I've also tried a few Android LLM apps, all running more than 30min.
Current LLM models are not running constantly on the phones to drain your battery. They just run when responding to a prompt. It by no means consumes more battery than a heavy game.