Hacker News Reader

Comment by tensor | Hacker News Reader

Neural networks really are pretty magical for a lot of image based problems these days. The impact for text problems is so far decidedly not magical, though there are starting to be some tantalizing results in my opinion.

View on HN

I'd say this is more of a result of decades of trial and error in different architectures/topologies rather than some fundamental property of neural networks in general.

More gimmick than magic. At best they can do a crappy job of emulating specific human behaviours on carefully curated training datasets.

What about Alpha Go Zeros approach of reinforced learning based on self play? Arguably the game of go has really simple, explicit rules, and evaluating the final end state is also easy, but it shows that with enough compute NNs can identify important features without supervision or labeling.

What about what about

Given enough time, energy, and computing power, you could just precompute every possible outcome and call it a neural network. You'll be praised as a genius.

This is like saying "given enough time, energy, and computing power, you could just factor large semiprimes by exhaustive search." It's both true and uninteresting because nobody has or ever will have that much time, energy, and computing power.

Ah yes, I forgot. Deepmind just secretly computed all 10^100 board states for alphago...

Neural nets have been very successful for a lot of things while also being overhyped, but dismissing it all as marketing hype is throwing the baby out with the bathwater.

Where "enough" is "more than is available in the universe". The number of outcomes you'd need to precompute exceeds the number of atoms in the universe.

Arguably the game of go has really simple, explicit rules, and evaluating the final end state is also easy, but it shows that with enough compute NNs can identify important features without supervision or labeling.

AFAIK we don't know how much of that generalizes.

> At best they can do a crappy job of emulating specific human behaviours

...With better performance than human, that is pretty magical.

In certain cases, perhaps. But that's the exception, not the rule. You can't simply apply a neural net to every problem and magically get great results.

Reply on news.ycombinator.com