Hacker News Reader

Comment by exabrial | Hacker News Reader

Without the ability to spin Osx instances up on the cloud, I don't see Metal making a dent in CUDA.

The cloud is staggeringly more expensive than building your own workstation. Like a couple of weeks of on-demand GPU instances on AWS Virginia costs about as much as a machine with 2 RTX 2080 cards in it that will stay performant for at least a couple years.

But coming back to the Mac Pro, unless you need Apple-specific software, the same is true compared to Ryzen/ThreadRipper + NVidia cards. The Mac costs 2-3x as much while delivering less performance.

At least the cloud can scale horizontally, the Mac Pro is just slow and inefficient.

Why is that? Shouldn't competition drive cloud prices closer to ownership prices?

There’s a high cost of entry (onboarding lots of customers, building data centers, writing the management software, etc.), so there isn’t much competition. The market just isn’t efficient in this case.

I think it does for stuff like storage, but transactional and heavy data processing are still cheaper to do on-prem from what I've seen.

Its because NVIDIA requires Teslas in the cloud, which while more powerful than 2080 ti's, cost roughly 6-8x as much.

How does that work? If I buy a 2080 Ti and then decide to use it in an internet-hosted box that I rent out to other people, the first-sale doctrine should tell Nvidia to piss off if they have a problem with it.

Because nvidia's driver licensing terms require you not to be using the consumer stuff in a datacenter.

What if you're running Linux with the nouveau driver?

It's irrelevant. You need the CUDA SDK and the CuDNN SDK both of which have license agreements.

Edit: You may also need the official Nvidia driver. I have never run anything with the OSS drivers.

Gotcha

When you are operating at any kind of scale you need the Telsa packaging (airflow, connectors, power) and testing (running many in a system). The consumer desktop cards cause lots of trouble.

Also the consumer cards have artificially small memory sizes (e.g. 11GB max) which painfully constrains DL jobs.

No manufacturer is allowed to build Tesla-like cards. Theoretically AMD could crush the nVidia profit margins by releasing cheap data center boards, but their developer support is rubbish and they want that high margin cash too.

.. what this means, is that Apple is competing against the Titan/NVIDIA hardware designers to make better compute power available at scale, and in a way which makes sense across consumer/pro boundaries.

In that context, the Mac Pro doesn't sound too bad a proposition. I say that as someone who recently built a dual-Titan/AMD Ryzen system, and while the pain of the build is almost gone away .. I do lust after that sexy Apple box, being plug 'n play and all ..

Intel competes with nVidia on high end GPU hardware and so far (KL and KF) are doing an abysmal job of it, despite using their CPU control to lock nVidia out of the CPU bus.

Apple can't really compete except in mobile GPUs, which is all about power constraints set to single digits Watts, vs >100W for server. That isn't all bad for Apple, there is lots of inference to be done, but by refusing nVidia hardware they impose a developer hurdle.

I'm looking forward to people putting 2080s into the new Mac Pro and seeing how reliable it is.

Nvidia wouldn't have prohibited people from using GeForce cards in data centers if people weren't do it.

People do try, but it isn't the basis for a successful computing platform. Works for block chain (though AMD delivers more flops/$ for simple sha-ing grunt work) and specific workloads where you have time and expertise but not much cash.

As for nVidia banning it, if you've the cash you can buy cards, just becomes painful. nVidia were doing it more because the GPGPU demand was choking the supply to graphics customers, and later dumping obsolete (for crunching) cards on the market en masse, interfering with the mid-tier card sales channel.

I believe they get around this by applying the restrictions to the software licensing for the CUDA libraries you have to download separately when installing things like TensorFlow.

The average corporate IT department is rubbish at doing computing. Easier and quicker to outsource than doing it in house. But nVidia sell GPU appliances (DGX) for these on-prem customers.

Basically, expertise is expensive and comes in $200k/year increments. You can use a lot of cloud GPU for that much money.

If you are doing serious training with on-demand instances you are doing it wrong.

If your training takes more than an hour, you really need to checkpoint it periodically. If you have periodic checkpoints you can use spot instances and pay 10% or less of what on-demand would cost.

1) Even with checkpoints and spot instances, the initial REPL development process needs to happen somewhere.

2) Where are you seeing 10%? More realistically spot instances are 25-30% of the cost. And if so fine, 6 weeks of machine time == dedicated workstation.

Yeah, one workstation. If you need to build out 1,000x as many GPUs, the cloud becomes more economical. And that's where the money is, in large customers, not someone who needs 1 or 2 cloud GPUs.

You forget that using the cloud you can spin up 9 ladies to have a baby in a month, or 270 ladies to have a baby in a day.

Agreed. I expect Metal to be used by game engines that target iOS, but there's way too much hardware tax involved for it to supplant CUDA or OpenCL for GPU programming workloads.

MacStadium exists.

MacStadium is really great, but it’s relatively expensive for a given spec and not as simple/agile as some other bare metal platforms. Nothing against MacStadium, though, they seem to do all they can to drag Mac kicking and screaming into the data center.

I was a MacStadium customer for years, and I really liked the products, but they are very expensive.

Reply on news.ycombinator.com