Google and Apple are going to move consumers and ordinary business users to edge inference. This is more disruptive to the upcoming IPOs than most investors understand.
They are just wildly outdated for most tasks. Even projects like frigate suggest buying different hardware. Coral was released in 2019 and hasn't been updated once. With a jetson nano or even a raspi ai hat you get way more tops
Ok I'll clarify: I want to buy a current-ish generation of TPU that can run LLMs. Coral is like 8 years old now
I've been really hoping to get Whisper running on a coral, but it can't even do that. There was recently some progress getting it to export to tflite (and even an official example added) but the coral is missing some instructions.
You can buy a Google Coral TPU, does that count?
With them being released in 2019 with zero hardware revisions they are just outdated for almost every workload
I would love to see them move off into edge computing. There should be some autonomy and siloed experience in day to day usage.
Yeah right, try generating an image on Apple Silicon or an Android phone with any of the Qwen or Flux family of models, let alone videos or prefill heavy LLMs.