Hacker News Reader

Comment by KerrickStaley | Hacker News Reader

Question: if I'm learning about neural networks and want to e.g. train a network to recognize MNIST digits, do I need a discrete graphics card (probably attached to a VPS that I would rent)? Or can I use the i5 Kaby Lake (which has an integrated GPU) in my laptop to train my network?

View on HN

To train an NN to recognize MNIST digits you need any cpu made in the last 5 years. Last 10-20 years if you want to wait a little longer. MNIST recognition was first done by convolutional neural network in the late 90s.

With less than 1% error. Here's a link to the paper: http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf

As other people have said, for MNIST, CPU is fine. One core. You can get over 90% accuracy with a network with just 1 hidden layer with only a few seconds of training. Or something like that. If you've never played with MNIST before, it's kind of amazing how easy it is. For instance, the following idea "works", in that you get results that are pretty bad, but much better than chance. (Even more than 50% right, I think.) Suppose you have 10 arrays:

zeros: each element of this array is a picture of a "0" ones: each element is a picture of a "1" ... nines: each element is a "9"

Compute the average of each array: the "average" 0, the average 1, and so on. Then classify new digits based on which average element they are closest to, using Euclidean distance. This sounds way too dumb to do any good, but the MNIST digits are normalized so well that this actually does something.

Even better: you can have a neural "network" that has zero hidden layers. This actually achieves almost respectable performance, believe it or not.

A 0 hidden layer NN is just a linear (or logistic) regression.

On MNIST I think you get something like 60% accuracy.

Yes, it is just logistic regression. I just dug up my old code from a few weeks (months?) ago, and it seems that somewhere around 90% is not too hard. 60% is way too low, unless I'm measuring wrong somehow.

MNIST is a toy problem. It shouldn't take you more than a few days or even hours to completely understand it. Do yourself a favor and get a GPU because soon you will want to try bigger networks on real world images.

1060, 1070, or 1080Ti are all good choices, depending on your budget and ambitions.

You actually don't even need the iGPU for simple MNIST training. The CPU is enough. Check out the sample MNIST training example with Tensorflow.

If you are asking about using the iGPU for a bit more power, most frameworks don't support them. The only frameworks with which you can use with iGPUs are which have OpenCL support, like caffe[1] (even for those you need to have a recent enough CPU)

Tensorflow either uses CPU or CUDA (Not even AMD, but the support is coming for openCL)

[1]:https://github.com/BVLC/caffe/tree/opencl

Simple network form tensroflow's MNIST tutorial trains in about 30 minutes on my macbook pro.

You can buy a 1 GPU machine with an i5 for $2,450 from Lambda Labs.

Reply on news.ycombinator.com