Having to make an API call to run your mission-critical robot seems like a non-starter.
Yeah, it’s surprising that this is the runtime model they went with as opposed to a hybrid edge/cloud model. I’d love to see a system 1 on edge, system 2 on cloud offering. They’re likely the best positioned to do this.
I agree, but isn‘t it basically impossible to do any kind of IP protection for LLMs, if someone dumps the disk or RAM - they have your weights.
I wonder how the cloud players get around this, maybe Antrophic trusts AWS and Google not to look, but do they really just trust Musk?
Bring your weights in an AWS-snowball-like briefcase with an armed guard handcuffed to it.