Training deep spiking neural networks is possible as well [3].
[1] http://compneuro.uwaterloo.ca/research/nef.html [2] https://www.nengo.ai/ [3] https://arxiv.org/abs/1611.05141
Training deep spiking neural networks is possible as well [3].
[1] http://compneuro.uwaterloo.ca/research/nef.html [2] https://www.nengo.ai/ [3] https://arxiv.org/abs/1611.05141
First, you only need a handful of transistors to implement a single spiking neuron, whereas digital circuits require ten to hundred thousands of transistors for corresponding numerical computations.
Second, spiking neural networks are intrinsically asynchronous. All the computing elements (neurons) are independent and only communicate via rare one-bit events (spikes). Correspondingly you (if any) only need a clock for the event bus but not for the vast majority of the chip.
See http://web.stanford.edu/group/brainsinsilicon/documents/IEEE... for a high-level overview.
Edit: To clarify: the asynchronicity argument is valid for digital implementations of spiking neural networks as well, especially as you scale up to networks of a million (or more) neurons. See http://apt.cs.manchester.ac.uk/projects/SpiNNaker/ for an implementation.
However, when implemented in a modern semiconductor process, it is relatively hard to implement analogue computations with negligible noise; especially when connecting distant parts of the chip. Spiking neuron models have the advantage of producing digital output (either they are spiking at a given moment in time or they don't). Thus this pulse can be easily transported without information loss, and the analogue computation is confined to a relatively small region, which makes it possible to reduce noise in the internal analogue signals.
A different way to think about it: say you wanted to implement an analogue model of a classical neuron (as used in deep neural nets) and wanted to transfer its output as a digital signal to mitigate the noise problem. In that case, the required analogue/digital converter would be far more complex than the analogue neuron itself. As you continue to reduce the size and precision of the A/D converter you'd end up with a 1-bit delta-sigma A/D converter, and you're circuit would very much look like a spiking neuron.
Edit: Reformulated some parts for clarity
What makes you think short pulses are more noise resistant than long ones? And if you try to utilize interspike timings to encode information you expose yourself to a whole bunch of additional challenges (not sure if those timings are used in the current spiking network models though).
Note that I'm only a layman when it comes to analogue neuromorphic hardware implementation details, I encourage you to have a look at [1] for more detailed energy computations.
[1] http://web.stanford.edu/group/brainsinsilicon/documents/IEEE...
Edit: Added a "significant" above. There indeed are minor problems with cross talk causing additional spikes, but mostly in the analogue neuron subthreshold regime, not in the AER bus. [2]
[2] http://journal.frontiersin.org/article/10.3389/fncom.2017.00...
I've looked at the NeuroGrid link. It's still not clear to me why they want to use spikes. Some parts of their design decisions (e.g. AER bus) are forced by the need to deal with spikes, and it feels like their hardware architecture would be simpler and more efficient if they didn't. Is the only reason "biological plausibility"?
Ultimately, what we want is hardware that can do what my 250 Watts 1080Ti card does (e.g. reach high classification accuracy on ImageNet) but faster and using less less power. Spikes don't seem like the best approach to achieve that.