Comment by baybal2 | Hacker News Reader

Comment by baybal2 | Hacker News Reader

Parent Full thread

>The problem was that you could instead wait for a couple years and x86 would be just as fast.

This still holds for the majority of users. Only tier 1 dotcoms can run stuff on GPUs economically. For everybody else, just getting many times, possibly few magnitudes, more commodity x86 machine time still makes sense.

I always remember the saying of my ops unit head once I worked in an advertising network. It was something to the meaning of: "a new coder like you who can find a fancy algo to do thing A and B faster pays off in two years, but a 10 new servers will pay off in just 2 month"

And I am still convinced that fabs will continue mercilessly squeezing CMOS till the last drop of blood for at least a decade. There are tons and tons of avenues to make cells smaller, cooler, and more performant that don't involve further process down scaling or switching to a next gen semiconductor.

> Only tier 1 dotcoms can run stuff on GPUs economically. For everybody else, just getting many times, possibly few magnitudes, more commodity x86 machine time still makes sense.

Disagree. Anybody running appreciable ML workloads buys and racks their own GPUs.

The fact that they buy and own their GPUs already says that they spent a whole magnitude more than if they just rented CPU time.

Moreover, say, a tier 2 player like a major AdNet rents its CPU time at rates that make AWS users cry. For them, it makes even more sense to just "let the code be a horrid contraption written by 10$ per hour outsourcing shop, but we can run in on cheap hardware"

There are other considerations beyond a simple cost per minute average here:

* Uptime guarantees - AWS goes down. Running your own hardware locally, you at least have a recourse.

* Storage and bandwidth - some of these datasets are huge and not amenable to streaming onto a temporary rental.

* Security - the ever-present bogeyman of cloud services. It might be harder to lock down your own machines, but it's the only way to fulfill some contractual requirements.

For commodity processing that doesn't involve these specialties, sure, fire away. That's what it's good at, and what most people need most of the time. But there's always a final TCO number and arriving at it requires accounting for all the risks.

Reply on news.ycombinator.com