1. Most useful LLM work is done in parallel. A Mac Mini can run one LLM inference thread at a time. The cloud can spool up dozens and spread that inference across efficiently batched operations over a fleet of hardware.
2. Faster inference hardware such as the chips from Cerebras and Groq cannot be run locally. But the advantages of running >5x the token throughput per thread can’t be overstated. Add in the multi-threading advantage and it’s a knock-out punch for local LLMs.
Local inference has a role: if you’re working with extremely private matters or you want an uncapped model that will talk dirty or generate NSFW photos, local is the only option. I think Apple and others will continue to also run a lot of useful workloads locally such as text editing suggestions, speech to text, text to speech, and image manipulation. As local hardware improves, these capabilities will get better too.
But, for most LLM work, the cloud will continue to dominate for a long time to come, if not forever.