Not cloud costs, though; consumer costs. These agents must run on-device for privacy protections, because anything less will be a global non-starter. That means we need models that excel on phones, tablets, laptops, and desktops; we need home compute modules to run larger, “frontier”-class models for bigger tasks but without guzzling electricity (the Mac Mini’s power envelope is an ideal target); we need smart harnesses that can foist off inference to bigger models elsewhere on your private network of devices as needed without the user prompting for specific thinking modes or model names. All of this has to happen automatically, privately, securely, without leaking data to advertisers or surveillance companies.
It’s hard work, but it’s definitely the future of AI. I would argue the business case is solved with being able to build bespoke stacks of kit and run models through neocloud providers with ZDR policies. It’s the consumer angle that’s hard to crack, because everyone wants a repeat of Web 2.0 for ad-dollars instead of a privacy-centric modern tech stack that forces companies into lower revenue streams.