Hacker News Reader

Comment by jdonaldson | Hacker News Reader

The brain has a number of functional parts that we don't understand all that well. Research on the brain hits a wall every now and then, but you never hear the phrase "Neuroscience Winter".

We're starting to train models that match biological brain behavior, at least in some crude functional/structural sense. Maybe this is just a string of coincidences, but my guess is that discoveries of analogous biological/model components will continue to happen, and we'll be able to learn more about AI and the brain by linking related phenomenon in vitro and in silica.

A few more recent examples:

https://deepmind.com/blog/grid-cells/

https://www.nature.com/articles/nature04485

View on HN

Neuroscience has been in constant winter, mainly because the methods are too crude and small scale, and that by observing 100 neurons out of billions makes it impossible to tell the whole story. The deep mind papers are interesting, but they are just a start imho. It may be that they are focusing on a mere coincidence. Nevertheless it is exciting to see progress in that direction, thats why i don't think DL research is going to lose steam soon. It would require a major show-stopper discovery (like the minsky-papert paper). Tesla crashes are no such thing.

It seems CS people are sick and tired of the hype, but i feel neuroscientists are now warming up to it.

I'd like to see neuroscience take more of a fundamental role in grounding/situating deep learning approaches. VGG is often mentioned as being roughly analogous to the visual cortex, but it differs in important ways. There's all kinds of why questions there. E.g. Why does deep learning work better with (e.g. ReLU) activation based on pooling while biological models use an inhibitory mechanism? Theory on why certain activations work better than others in DL is a little weak imho. Right now ML practitioners just throw a lot of parameter combinations at the wall and see what sticks. That's fine, but it's not really indicative of a robust understanding of model behavior.

I 'm actually of the opposite idea - let the two fields evolve by their own darwinian process as this will yield more interesting results. DL itself has created its own scientific questions and puzzles that may lead to important discoveries (which may transfer to neuroscience). E.g. "why does batch normalization / dropout work."

Reply on news.ycombinator.com