Hacker News Reader

Comment by deepnotderp | Hacker News Reader

Perhaps more efficient memory use is the answer instead. This article has mentioned most of the published techniques.

As for the 1080 Ti, a better option might be the P100, which can have up to 16GB.

A better option... if someone else is paying for it. It is literally ~10x the price of a 1080ti ($700 vs a suggested $7000).

The P100 has much better double-precision performance, but it's single-precision perf doesn't justify the cost in most instances.

The DGX1 (w 8 P100's) is another example of an overpriced, turnkey money extraction "solution."

Also worth noting it's hard to get a Ti non-founder edition, which has a hefty premium on it for the privilege of being first. The regular 1080 non-FE cards are probably the best bang/buck right now.

Folking doing ML/AI/DL, or any HPC really, gotta profile their apps to find bottlenecks and root out resource waste: single prec math? ram? algorithm complexity? storage iops? network latency? etc. because throwing money at a problem is no substitute for having a clue how to use money, time and electricity wisely.

Compared to what it can replace DGX-1 is not overpriced in any way.

Feel free to invest your time and money into building and debugging issues on your own system. Companies which pay big $$$ to their data scientists have different incentives than regular users - you do not want to waste time of your expensive staff on chasing hardware/software issues.

129000$ for the DGX-1, for anyone interested.

Saves you debugging your way to a workable 8*1080ti, when you look at it from this angle. Not sure if being able to swap out the cards is what makes building them yourself worth it in addition to the cost saving.

P100 has double rate FP16 which makes it twice as fast, so make that 16x 1080Ti. Also those won't fit into one machine, so now you're building a cluster, and what are you going to use for the interconnect? You won't get linear scaling, so make that more like 24x 1080Tis plus extra development effort for scaling on a cluster, if your problem even scales that way. Now account for the power usage...

There is also the unified memory on the DGX-1, which is supposed to make streaming data to/from the GPUs a lot more efficient, as well as enable much larger data sets.

What interconnect do you have between those 8*1080ti? If your problem doesn't require communication then it may make sense.

If you require non-trivial communication no PCIe setup would beat NVLink communication bandwidth on DGX-1.

> 129000$ for the DGX-1, for anyone interested.

If you are allowed to buy one. Nvidia doesn't have nearly enough chips to meet demand, the bulk of their production has been earmarked for quite a while ahead, what little is left over and makes it into the DGX-1 is sold to very few hand picked customers.

The P100 also has 2x the performance on FP16 and NVLink, which is about 10x faster than PCIe.

> As for the 1080 Ti, a better option might be the P100, which can have up to 16GB.

$4K or so, vs $700?

Most of the major labs I've seen pay six figures easily to their top researchers. Good compute is one of the biggest reasons why researchers would want to work somewhere. Skimping on compute seems foolish at that stage.

Here's a half-joking, half-serious take on the matter: https://twitter.com/deliprao/status/842636496666419200

Yeah but is that P100 better than five or six 1080Ti cards?

Because parallelization is so difficult, yes.

If you are using a GPU, parallelisation is not difficult.

I believe you may be missing something. Even on a GPU there are synchronization points in some algorithms and data needs to be passed from one processor to another once you spill over the boundary of what can be held in the RAM.

Also, some parts of an algorithm may not be parallelizable at all.

GPU parallelization is only 'not difficult' if:

  - all your data fits on a single GPU

  - your code is embarassingly parallel
 
  - for the total duration of one computation

Bonus if you do can use the output of one computation as the input of the next. In all other kinds of computations the usual bottle-necks apply.

https://en.wikipedia.org/wiki/Amdahl%27s_law

I was talking about multi-GPU parallelization. Multi GPU SGD parallelization is not easy. Single GPU parallelization is trivial of course.

> Most of the major labs I've seen pay six figures easily to their top researchers.

Even a normal programmer easily makes six figures, I hope they are doing better than that given the demand in this space.

That was a conservative estimate.

I thought you might have meant monthly

Reply on news.ycombinator.com