Hacker News Reader

Comment by varelse | Hacker News Reader

By the time any of this is viable commercially, I really wonder where commercial GPUs and accelerators will be.

And they're going to have the home team advantage when that happens. So that means that unless these things are as accessible to tensorflow/pytorch or whatever the heck the framework de jour is at that point (hopefully more like Jax but better), they will get no traction.

Evidence? Every single demonstrably superior CPU architecture that failed to dislodge x86 over the past 50 years. Sure, it's finally happening, but it's also 50 years later.

View on HN

Luckily chip companies know this now and all the alternative deep learning accelerators have some level of support for mainstream frameworks. Getting their support upstreamed to the main framework project is another matter, though, as it reaching the quality of implementation as CUDA and CPU...

We've already seen disruptive architectures like Google's Tensor Processing Units. x86 has already been upended for ML, and photonics based processing units will simply be another PCI card you plug into your computer just like a GPU.

How disruptive are TPUs really though? My understanding is that essentially everything is still trained on Nvidia architecture.

If you design your network on a TPU, you will tend to use operators that work well on a TPU. And in the end you will have a network that works best on the TPU.

Lather rinse repeat for any other architecture. You can even make a network that runs best on Graphcore that way, but it won't be fun to do it. You might even get Graphcore to pay top dollar for it though as they both need some good publicity and they have lots of VC left to squander.

This also tends to be true of video games where the platform on which they were developed is the best place to play them rather than their many ports.

We’ve been doing this for years with DSP and networking. So kind of ho-hum from a HW perspective.

If you ask me the thing that makes these things even remotely interesting is the willingness from the SW side to support new HW architectures. Without that you can’t have any innovation in HW.

That's only because google is stupidly refusing to sell their devices

Not to mention it is only happening due to the emergence of mobile with a different architecture (new field, existing moat is meaningless etc etc)

Better CPU arches were never faster, definitely not by much.

Going to disagree. I ran circles around contemporary x86 back in the early '90s because of specific instructions Intel denied they would ever need in their processor roadmap. But it really didn't matter and that's one of the most important lessons of my career.

They did similar goofy thinking with respect to the magic transcendental unit on GPUs so it's not like they ever learned. It's not entirely about clock rate.

Reply on news.ycombinator.com