You could, but it might not be particularly useful. They already have a really good simulation environment and ways to connect it to virtual environments. It would be useful if you wanted to extend the model with further neural populations!
198 karma · joined October 26, 2015
You could, but it might not be particularly useful. They already have a really good simulation environment and ways to connect it to virtual environments. It would be useful if you wanted to extend the model with further neural populations!
> unfortunately docs aren’t helping at all
Yeah, the docs really aren't the ideal entry point. Instead, I'd recommend installing the [Nengo GUI](https://github.com/nengo/nengo-gui) and going through the built-in tutorial.
> 1. Where can I find a diagram of neuron you use?
We use a bunch of different neurons [1], but the default one is the Spiking LIF. It's really common in computational neuroscience.
> 2. Where can I find a diagram of network you use (for simple example)?
Check out this video example in the Nengo GUI https://youtu.be/UVeIPKnQAL4
> 3. What is an example of simple learning algorithm for above?
There's a tutorial in the Nengo GUI, but there's also a few Jupyter Notebook examples: https://www.nengo.ai/nengo/examples.html#learning
[1] See related psychology.stackexchange.com question https://psychology.stackexchange.com/a/9131/4397
Calling the answers "partial" is fair, but I feel like the author understates the number of observations backing typical neural models? The lab I belong to created Spaun, which admittedly uses a pretty simple neuron model, but still matched a ton of neural and behavioural data! There's also Leabra, which I have some qualms with, but still has a pretty large collection of results.
That being said, I feel like they're over-selling the capabilities of reservoir computing. Yeah, [you can stack them](https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2016...) and get pretty high accuracy on a few tasks, but it's still not competitive with traditional Deep Learning.
Update: read the paper and they cover a few examples in it. Don't have enough domain knowledge to judge whether the claims are accurate, nor whether they generalise to something like clash-lang.
Edit: linked to specific post in forum thread instead of the whole thread
Very personally, I see it as contributing a small amount of a very non-scarce resource (bandwidth, disk space, computing power) so that I have greater control over the platform I'm using. I loved both Twitter and Medium, but the corporations that run them have different ideas about what they're for. I see web decentralised about letting the user choose their experience instead of having it enforced on them.
tl;dr doing pure Deep Learning (also known as connectionist) models of the brain limits your tools in a bad way. Using tools from Dynamicism and Symbolicism is better. As proof, check out Spaun, the world's largest functioning brain model.
Note: I mostly just disagree with him philosophically, in terms of his reasoning because it's overlooking some evidence. Don't really have an option on his conclusion. Probably agree with him more than I disagree with him.
Depends on what you mean by "well". It would be foolish of me to deny the advances of deep learning. However, there are limitations and the NEF can act as a bridge in those cases. For example, if you want:
1. To approximate a dynamic system 2. To use symbolic computation 3. Use the neural networks as an initial guess at a solution to a problem and then want to use gradient descent to figure out the actual solution
I talk about using NEF as a bridge in [this blog post](https://medium.com/@seanaubin/deep-learning-is-almost-the-br...), if you're curious. Unfortunately, I don't know of any examples using the "initial guess" paradigm that I describe, but that's more of a "not enough people using Nengo" problem.
There is if: 1. You want an energy-efficient distributed representation of the function. 2. You want to use the function as an initial guess and then learn online afterwards.
There are two ways to build spiking neural net architectures without training. The [Neural Engineering Framework](https://medium.com/@seanaubin/a-way-around-the-coming-perfor...) and [Deneve's approach which I'm more uncertain about](https://forum.nengo.ai/t/robustness-of-deneves-networks/320?...).
To be clear, when I say "training" in this context, I mean they don't require iterative rounds of stochastic gradient descent. They just need to be given the function that you want to approximate and then they're done.
[1] https://github.com/arvoelke/delay2017/blob/master/delay2017....
As for Liquid State Machines, those evolved into Reservoir Computing and Echo State Machines. If you want to read more about them, I would recommend this paper comparing them to the Neural Engineering Framework [4] (with code!) to get a good idea of the state of the field.
[1] https://medium.com/@seanaubin/deep-learning-is-almost-the-br...
[2] https://cogsci.stackexchange.com/q/16269/4397
[3] https://medium.com/@seanaubin/a-way-around-the-coming-perfor...
[4] https://github.com/arvoelke/delay2017/blob/master/delay2017....
Numenta isn't biologically plausible [1], so although the resulting networks might be mappable to the brain in the same manner as Deep Learning and FORCE trained networks, there's no way to map the learning period onto brains.
[1] https://forum.nengo.ai/t/hierarchical-temporal-memory-vs-nef...