Comment by zerostar07 | Hacker News Reader

Comment by zerostar07 | Hacker News Reader

An AI "winter" is a long period in which [edit: funding is cut because...] researchers are in disbelief about having a path to real intelligence. I think that is not the case at this time, because we have (or approaching) adequate tools to rationally dismiss that disbelief. The current AI "spring" has brought back the belief that connectionism may by the ultimate tool to explain the human brain. I mean you can't deny that DL models of vision look eerily like the early stages in visual processing in the brain (which is a very large part of it). Even if DL researchers lose their path in search for "true AI", the neuroscientists can keep probing the blueprint to find new clues to its intelligence. Even AI companies are starting to create plausible models that link to biology. So at this time, it's unlikely that progress will be abandoned any time soon.

E.g. https://arxiv.org/abs/1610.00161 https://arxiv.org/abs/1706.04698 https://www.ncbi.nlm.nih.gov/pubmed/28095195

No, AI winter was when the AI people oversold the tech, then failed to deliver, and lost their funding. This is well documented in histories of the field.

I think the scientific pessimism preceded the funding cuts:

https://en.wikipedia.org/wiki/AI_winter

The brain has a number of functional parts that we don't understand all that well. Research on the brain hits a wall every now and then, but you never hear the phrase "Neuroscience Winter".

We're starting to train models that match biological brain behavior, at least in some crude functional/structural sense. Maybe this is just a string of coincidences, but my guess is that discoveries of analogous biological/model components will continue to happen, and we'll be able to learn more about AI and the brain by linking related phenomenon in vitro and in silica.

A few more recent examples:

https://deepmind.com/blog/grid-cells/

https://www.nature.com/articles/nature04485

Neuroscience has been in constant winter, mainly because the methods are too crude and small scale, and that by observing 100 neurons out of billions makes it impossible to tell the whole story. The deep mind papers are interesting, but they are just a start imho. It may be that they are focusing on a mere coincidence. Nevertheless it is exciting to see progress in that direction, thats why i don't think DL research is going to lose steam soon. It would require a major show-stopper discovery (like the minsky-papert paper). Tesla crashes are no such thing.

It seems CS people are sick and tired of the hype, but i feel neuroscientists are now warming up to it.

I'd like to see neuroscience take more of a fundamental role in grounding/situating deep learning approaches. VGG is often mentioned as being roughly analogous to the visual cortex, but it differs in important ways. There's all kinds of why questions there. E.g. Why does deep learning work better with (e.g. ReLU) activation based on pooling while biological models use an inhibitory mechanism? Theory on why certain activations work better than others in DL is a little weak imho. Right now ML practitioners just throw a lot of parameter combinations at the wall and see what sticks. That's fine, but it's not really indicative of a robust understanding of model behavior.

I 'm actually of the opposite idea - let the two fields evolve by their own darwinian process as this will yield more interesting results. DL itself has created its own scientific questions and puzzles that may lead to important discoveries (which may transfer to neuroscience). E.g. "why does batch normalization / dropout work."

That’s not what the AI Winter was. It was when anything that used the term “AI” (including academic research) was unfundable.

From what I’ve read of the last AI winters, funding was more centralised, & projects were larger and fewer. Hardware was scarcer.

These days anyone with a few dollars to spend on compute time and some free software can do machine learning. I can’t see that going away.

Reply on news.ycombinator.com