Hacker News Reader

Comment by rjtavares | Hacker News Reader

> I know not every researcher or entrepreneur has the resources of NVIDIA to train for this many days, but let's not forget, that part needs to occur only once.

It's not like they trained this on a GPU farm. According to the paper [1], they "trained the network on a single NVIDIA Tesla P100 GPU for 20 days".

[1] http://research.nvidia.com/sites/default/files/pubs/2017-10_...

View on HN

Yes, but they probably did not get it right on first try. You usually need hundreds of iterations to get it right. This does not imply 2000 days of GPU, but it quickly pushes this outside of reach of consumer hardware.

Where I live that thing alone is $17,000.

P100 is essentially 1080Ti, so you can grab 1080 Ti and have the same speed. V100 might be the expensive version, up to 10x faster.

You should look at GPU memory bandwidth for a proxy for performance when training DNNs. The P100 is about 40% faster than a 1080Ti. The V100 is only about 75% faster than a 1080Ti.

Based on this, i expect these commodity GPU servers (with 10 1080Ti cards) that cost 1/10th of the DGX-1 will be huge: https://www.servethehome.com/deeplearning11-10x-nvidia-gtx-1...

It may be the same hardware, but nVidia is known to block existing hardware features in their drivers on consumer level chips to drive prices. They did this in the past with quad buffer stereo (full screen only, not available through OpenGL) or decreased double precision performance (transparently added NOPs after each double precision opcode).

I will believe that these devices are equal when someone shows me benchmarks proving that. Until then, I am skeptical based on past experience.

Reply on news.ycombinator.com