I guess I should say to be fair, getting the user input and model output onto a single set of hardware with even semi-reasonable guarantees that it won't be examined in an unencrypted state is a heck of a lot better than what OpenAI is doing, so I don't necessarily want to say it's a bad business model -- there are lots of services I rely on that make privacy promises to me that aren't based on complete E2EE setups. What you're describing is absolutely an improvement over OpenAI's privacy.

I think my quibble is that (to me) the privacy terms on the homepage feel like they're suggesting something stronger than a secure enclave. My first thought when I saw it was "huh, somebody figured out homomorphic encrypted models." I'm not sure I'd have the same objections if I hadn't gotten that impression to start with.

----

My suspicion with local models is less that they'll run on a cell phone or a low-end laptop, and more that at the point they can run on high-end consumer hardware, they'll also be cheap enough to run that hosting an LLM might become a commodity service. In the same way that a hospital might not want to host its own website or data records, I'm still wondering how the companies involved in that stuff don't eventually just turn into generic hosting companies. And again, to be fair here, if the security model you're talking about holds up, maybe that's a model that can be applied to multiple products or hosting offerings. It does sort of feel like the main thing you're advertising here is the generic enclave, not the AI specifically.

And who knows, maybe I'm just completely wrong about all of it, I'm not an expert. But I sort of suspect that it's going to get progressively easier to host these kinds of models in the future.