Hacker News Reader

Comment by modeless | Hacker News Reader

Contrary to what you might read in puff pieces from IBM PR, all existing "neuromorphic" hardware is terrible. The neural nets that work don't look anything like the brain, and neural nets that look like the brain don't work.

People constructing neural nets that look like the brain are engaged in cargo cult science, because the truth is we have no idea yet how the brain works and attempting to imitate it without knowing that is doomed to failure.

View on HN

I agree that neuromorphic is often specious, but it's not fair to say that neuromorphic hardware fails if it can do non brain like computations more efficiently than conventional hardware.

The thesis may be misguided at times but building non-traditional hardware that provide significant efficiencies is a wonderful thing. Sometimes it takes a random vector to get off a local maximum.

For example, did you know Synaptics, the guys who developed the touch input you're probably using started as a crazy neuromorphic thing from Carver Mead? [0]

While most of the IBM press is garbage, they do seem to be making progress running conventional DNNs on their hardware, showing backprop in low precision, and suggesting progress on CNNs. [1,2]

[0] https://en.wikipedia.org/wiki/Carver_Mead#Touch

[1] https://papers.nips.cc/paper/5862-backpropagation-for-energy...

[2] http://p9.hostingprod.com/@modha.org/blog/2015/12/nips_2015_...

This is off topic, but Synaptics touchpad drivers have been the worst part of using a Windows laptop for years and years. Microsoft finally had to take matters into their own hands and write touchpad drivers themselves in Windows 10 because Synaptics just couldn't do it right.

Back on topic, that paper is pretty interesting, though MNIST is a toy problem. If they could run any of the ILSVRC winners then I would be impressed, but I can't help but think that if one wants to run CNNs one would be much better off designing a CNN chip to begin with rather than a wacky spiking architecture and trying to shoehorn a CNN into it. To my knowledge nobody has yet fabricated an ASIC specifically for CNNs so there's no way to fairly compare their approach to something like that.

Both good points. Yann LeCun (et al) have developed CNN FPGA designs where they project future ASIC performance, but I'm not aware that this has ever been taped out, nor any power numbers released:

http://yann.lecun.com/exdb/publis/pdf/farabet-iscas-10.pdf

IBM has been working on these chips from 2009 I think, before DNNs really became popular again and I bet they would change a few things if they were designing from scratch today - especially better support for CNNs. The cost of chip production for something like True North is very high (5.4B transistors!) so I suspect they are trying to squeeze as much out of it as they can :)

I'm curious to hear your opinions on Numenta's work on emulating parts of the neocortex.

He just gave it to you. Their system is not really competitive in any metric people are measuring.

A good measure of any of the techniques are the results. We see record object recognition and speech recognition results. How many of these results are from Numenta or IBM neomorphic chips? Most successes have been on deep learning architectures over GPUs (and large datasets). While these fancy architectures may have their applications, they have nothing to do with recent advancements in last 3 years.

To be fair, the goal of those new computing architectures is not to advance algorithms. That at best is a side benefit.

Those algorithmic performance improvements seen in DNN have come from improved datasets and training. Numenta and IBM arent focusing on training AFAIK. Google's Quantum Annealing [0] is the only hardware I'm aware of focused on training, although there are rumors Nervana Systems may produce something [1]. I'm sure there are others; accelerating training of DNN isnt a particularly new idea.

The goal of these other computing architectures is typically to provide lower power, higher frequency/lower latency, or smaller form factor execution of trained models, but there is a question of how much value they can provide over more conventional chips to be worth the chip design costs.

However without these architectures becoming as mainstream as say a GPU, I think we will continue to see advances come from the typical everyday computer. The ML community seems to be much more democratic than others.

[0] http://googleresearch.blogspot.com/2015/12/when-can-quantum-...

[1] http://www.nervanasys.com/about/

Numenta benchmarks their algorithms, along with others, on real-time streaming data: http://numenta.com/numenta-anomaly-benchmark/

Reply on news.ycombinator.com