Hacker News Reader

Comment by kelnos | Hacker News Reader

Google said:

> And in our new Tensor G3 chip, every major subsystem has been upgraded, paving the way for on-device generative AI.

This is definitely weasel-worded, but note that this does not actually have to mean that all the generative AI stuff can be done on Tensor G3. They could (credibly, but, again, weasel-y) claim that the stuff they've done on G3 is prep work that is "paving the way" for a future chip to be able to do it all.

But either way, I'm not terribly surprised? Generative AI on a mobile SoC (even with special-purpose hardware) still sounds like a bit of a stretch at this point, no? At least, doing it on-device with acceptable performance and power consumption seems a little unlikely?

View on HN

There are apps on iOS that do run stable diffusion like models locally on device.

https://apps.apple.com/ca/app/draw-things-ai-generation/id64...

I believe the Tensor chip is powerful enough to do the same but I suspect the efficiency is low enough that they don’t want to run it locally.

That’s the difference between apple approach and google approach.

Apple does a lot of computation on your own devices (in the name of privacy and what not) and ends up saving a lot of server compute.

Googles strategy so far has been just burning cloud compute costs.

From what I understand google's hardware doesn't measure up to apple even when just comparing ML cores.

Google has an enormous advantage over apple in data center compute. For one thing, they don't have to pay the Nvidia monopoly tax to do inference. It's silly to color this as a disadvantage.

Not that surprising really when you consider where the companies come from and what their other major successes are.

This isn't a fair characterization, though. At least not as reductively as you're making it sound.

Google ALSO performs transcription/live captioning/speaker diarization on-device, for privacy, for instance.

The features being discussed here are the generative AI features, like Magic Editor and their heretofore not-yet-launch Video Boost mode.

I would say it's also done for copyright reasons. Google will not do assistant read aloud if the article is behind a paywall. But the new read aloud that is done on device allows for reading almost all any website.

why is this even the headliner complaint?

Post-launch embargoes to deceive consumers about actual performance is a MUCH bigger deal.

What is this in reference to? Sorry, OOTL on “post-launch embargoes”

From the article:

> As we covered exclusively earlier this week, in an extraordinary move, Google went out of its way to block reviewers from being able to easily install popular benchmark apps through its Play Store during the review embargo period. This actually also extended into the post-launch period too, however Google lifted the ban after our article went live. Tests using Primate Labs’ popular cross-platform benchmark Geekbench 6 showed that -- despite having quite new CPU architecture -- Tensor G3 performance is closer to the mid-range Qualcomm Snapdragon 7+ Gen 2 than it is to its current flagship chip the Snapdragon 8 Gen 2.

Reply on news.ycombinator.com