Durable storage + sufficiently performant local LLM hardware + appropriate models [last two are wildcards and hard to predict the forward curve imho] = this experience locally. Remember when mobile video hardware acceleration wasn't good? And now, it is very good.
> Character.ai's Bankruptcy Documents Show the Economics of AI Don't Work
My comments:
"I don't agree, here is why."
This specific model doesn't appear to be economically sustainable and/or commercially viable. That is a reasonable statement and assertion, based on the evidence presented. Does not mean the economics of AI in general do not work. That's a very broad statement! Nuance, words mean things.
Therefore, we should ask ourselves and discuss, "Under what circumstances could|would the economics work? Are there any? Or are there none?" We are discussing the problem space! Be more curious.
"eventually" is doing a lot of work here. Like sure, yes, eventually. But that could be later this year or it could be 50 years from now. We don't know the answer to that. Assuming it's going to happen in a timeline short enough to matter for this conversation isn't worthwhile. It's like saying eventually we'll have cold fusion, so why is everyone skeptical?
As I said, it may happen within a year. But maybe life takes a different direction. The person to whom I responded said "eventually", and yes, "eventually" it will happen. but that's not a useful time horizon to plan around because it could be a lot of things.
The hard part is refining the UX, imho, while supporting pluggable LLM support on the backend. If the future never arrives of cheap on device inference, still works if you pay as you go and for those users willing to pay for their usage directly through their cloud inference provider of choice or through the app (unless the future is "cloud inference is unsustainable at consumer price points and on prem inference never arrives"). Just my two cents.
This is fundamentally what Apple did with new Siri, powered by Gemini today. They will eventually flip the switch to an in house solution. Agree the wildcard that still remains is "can this be done on devices eventually?"
https://www.apple.com/newsroom/2026/09/siri-ai-a-profoundly-...
https://www.macrumors.com/2025/11/05/apple-siri-google-gemin...
> Apple weighed using its own AI models for the LLM version of Siri , and also tested options from OpenAI and Anthropic, but it decided to go with Gemini after deciding Anthropic's fees were too high. Apple already has a partnership with Google for search results, with Google paying Apple around $20 billion per year to be the default search engine option on Apple devices.
> Though Apple is planning to rely on Google AI for now, it plans to continue working on its own models and will transition to an in-house solution when its LLMs are capable enough. Apple is already working on a 1 trillion parameter cloud-based model that could be ready as soon as 2026. Apple is unlikely to publicize its arrangement with Google while it develops in-house models.
Specifically with regard to AI, the consumers aren't even aware of the "grocery store" either. They're being served from a silver platter. What matters most is the quality of results, not the methods. You can disagree all you want for whatever (good and compelling) reasons, but it doesn't change the fact that many people do not even buy groceries now and doordash everything.
If it turns out that you can put a mobile kitchen in their backyard that cooks up everything without them realizing, it makes no difference to the consumer.
Realistically, yes there's a finite amount of training data, hence the plateau in capability we've seen for at least a year. I think we're likely to see a consolidation and compression of all of this into local models. The economics don't support the expected growth of the current arrangement either. There simply isn't that much distinct data out there as initially assumed.
No-one's going to want to talk to a 7B model for more than 3 minutes
My assumptions are that inference costs continue to decline (and/or the difficulty level of routing requests declines) and hardware acceleration continues into the future. These assumptions might be wrong, but that is the idea the thesis is based around.
(have built workflows and harnesses that currently target LLM provider APIs under the assumption I'll be able to point "locally-ish" eventually, my career had a strong focus on distributed computing about 10-15 years ago, think managing data and work clusters for LHC detector data processing, and I grew up on hobbyist distributed.net and BOINC projects, all the lenses I see this problem through; queues, jobs, sync vs async performance optimization, etc; recovered sysadmin|infra/devops engineer a lifetime ago)
Significant investment is being spent on producing hardware that is more suitable for running mid-sized (100B-ish) models. Starting with flash chips with wide enough interfaces to provide the weights (NAND flash can be ~5x more energy-efficient for reads than DRAM), and going into more exotic solutions like compute-in-memory.