Computational Power Found in the Arms of Neurons
quantamagazine.org
quantamagazine.org
That's an idea, I believe, that has been perpetuated in the general public's mind by popular science articles and descriptions about deep learning and neural networks, in a (largely successful) attempt to attract and gain attention for the subject and those articles.
One thing that did capture my imagination from this new research, though, as applied to deep learning, is whether this would necessarily cause any change in actual deep learning models?
That is, even if the dendrites are performing more complex computations, does that change anything about the "standard" layered model of deep learning systems, or does one simply add more artificial neurons to the model to compensate (and thus, nothing is really gained)?
Finally - and I know this is not the focus of this research - this new understanding doesn't seemingly do anything to advance our knowledge of how actual brains and neurons learn; as mentioned elsewhere, our current artificial models implement "back propagation" for learning purposes. While that has been a radically successful method, even its creator (Hinton) believes it to be wrong (and is working toward a different model of so-called "capsule networks" in response). As far as I am aware, no form of back propagation has been found to be performed in natural brains or neural networks, so we really don't know how this aspect works. Plenty of competing theories and ideas have sprung up regarding this.
Hopefully one day we'll figure it out; I'm of the opinion that, like so many other things that have been discovered in the past, when we do figure it out, it will probably be something extremely simple and curious as to how it remained overlooked for so long.
I kinda have a thought or idea that its done via "societal feedback loops" - that is, brains talking/interacting with other brains - though how you integrate this with a feedforward system isn't apparent to me. I'm also certain, though, that this idea isn't something new or unique (and it very likely is wrong, too).
https://www.biorxiv.org/content/10.1101/613141v1.full
They model a single cortical neuron as a deep neural network with 7 hidden layers consisting of 128 hidden units each.
Biological neurons are A LOT more powerful than "neurons" in neural networks. If you hear a claim about computer-brain parity being close - the people making it almost certainly don't understand the power of cortical neurons.
The brain doesn't backpropagate. So how do you do it?
Then there is also the brains reward center, the one that famously goes wrong when you get addicted to drugs. It releases chemicals which reinforce behavior.
And finally some parts are "hard-coded" to be a certain way our DNA.
So yeah even without intra-dendrite learning there are biological justifications for what you propose.
[1] https://drive.google.com/file/d/1R4mS_4vzf5mycQez76O_mmpQDpT... (PDF)
Perhaps trying to grow something like a brain may give us an insight on the actual algorithm used.
https://www.newscientist.com/article/dn6573-brain-cells-in-a...