Hacker News Reader

Comment by ghaff | Hacker News Reader

>an era of more architectural diversity

We're already seeing this in machine learning. I used to follow the processor space fairly closely and I'd see various companies come out with designs that were specialized for some workload or other. The problem was that you could instead wait for a couple years and x86 would be just as fast.

You also have some other things going on like open source software, centralized computing in clouds, etc. that make having hardware that's optimized for some specific workload much more tenable.

View on HN

>The problem was that you could instead wait for a couple years and x86 would be just as fast.

This still holds for the majority of users. Only tier 1 dotcoms can run stuff on GPUs economically. For everybody else, just getting many times, possibly few magnitudes, more commodity x86 machine time still makes sense.

I always remember the saying of my ops unit head once I worked in an advertising network. It was something to the meaning of: "a new coder like you who can find a fancy algo to do thing A and B faster pays off in two years, but a 10 new servers will pay off in just 2 month"

And I am still convinced that fabs will continue mercilessly squeezing CMOS till the last drop of blood for at least a decade. There are tons and tons of avenues to make cells smaller, cooler, and more performant that don't involve further process down scaling or switching to a next gen semiconductor.

> Only tier 1 dotcoms can run stuff on GPUs economically. For everybody else, just getting many times, possibly few magnitudes, more commodity x86 machine time still makes sense.

Disagree. Anybody running appreciable ML workloads buys and racks their own GPUs.

The fact that they buy and own their GPUs already says that they spent a whole magnitude more than if they just rented CPU time.

Moreover, say, a tier 2 player like a major AdNet rents its CPU time at rates that make AWS users cry. For them, it makes even more sense to just "let the code be a horrid contraption written by 10$ per hour outsourcing shop, but we can run in on cheap hardware"

There are other considerations beyond a simple cost per minute average here:

* Uptime guarantees - AWS goes down. Running your own hardware locally, you at least have a recourse.

* Storage and bandwidth - some of these datasets are huge and not amenable to streaming onto a temporary rental.

* Security - the ever-present bogeyman of cloud services. It might be harder to lock down your own machines, but it's the only way to fulfill some contractual requirements.

For commodity processing that doesn't involve these specialties, sure, fire away. That's what it's good at, and what most people need most of the time. But there's always a final TCO number and arriving at it requires accounting for all the risks.

> centralized computing in clouds [could] make having hardware that's optimized for some specific workload much more tenable.

I think this is the key. GPUs are pretty much the only consumer-level specialized co-processors, and those have been selling like gang busters. Plot twist, it's the data centers that are buying them up... to the point that the manufacturers tried to prevent you from buying more than one at a time.

Of course specialized processors need to have proper networking and storage proximity to really deliver value. May be more challenging to offer a specialized service like this than you'd hope. Perhaps the big cloud vendors will dominate again, so chip manufacturers need to just hope for a few big contracts to Amazon / Microsoft / Google.

> Plot twist, it's the data centers that are buying them up... to the point that the manufacturers tried to prevent you from buying more than one at a time.

No, it's the cryptocurrency miners that caused that restriction to be introduced.

Not sure why you're being downvoted. It's possible that nVidia's unsold GPUs are just a coincidence with crypto price drops, but I'd like to see some evidence.

Reply on news.ycombinator.com