Hacker News Reader

Comment by deepnotderp | Hacker News Reader

This is literally just "whatever looked cool"... Where is alphago, neural arch search through RL, learning to play in a day, wavenet, and pixelcnn/rnn? That's just off the top of my head....

View on HN

Actually, I think alpha-go would be more in the "looked cool" category. It was an applied engineering task, not so much a fundamental approach change like adversarial networks. That's not to take away from the monumental nature of the feat, but it seems like this blog post was more about higher level developments. Similarly, pixelcnn and wavenet were "what can you do when computing power is no object".

I would have liked to see something on RL^2 / "learning to reinforcement learn" which do seem like huge developments to me, but maybe are too new to see the impact of yet.

I don't know. I see a lot of papers which are getting great results on tough domains like program writing using the idea of tree search guided by NNs which may be very old but AlphaGo brought to everyone's attention by demonstrating what modern NNs could do in it. Any domain which is tree or DAG structured and you have a simulator for can be tackled much more effectively now.

Heuristic search is a classic technique for getting things done in AI. Just wasn't good on recognizing the value of some moves. Aka scoring. Today's AI people have ANN's that are great at scoring. Their tunnel vision just made them focus almost exclusively using such tech. Then, a team combines two proven methods to do what each are good at. Big results follow.

These fads and tunnel vision happen all the time. The most interesting stuff often happens when a talented person inside the bubble opens their mind to bring in something outside it.

I agree with this; I've seen people writing off AlphaGo as if it was very little new conceptually over MCTS (but just with more hardware). But the power of the NN was surprising.

There's a paragraph from the AlphoGo paper that I think speaks to this:

"We also tested against the strongest open-source Go program, Pachi, a sophisticated Monte-Carlo search program, ranked at 2 amateur dan on KGS, that executes 100,000 simulations per move. Using no search at all, the RL policy network won 85% of games against Pachi."

I think it's more a demonstration than anything, but unlike these demonstrations, no one expected just how powerful deep nets could be. No one expected they could win Go, by contrast, everyone (in DL) expected that they could do these things.

I do agree though that meta-learning is the future and very interesting, especially interested in the fact that DRL can now construct nets better than humans (and you can actually speed it up with an in-house technique I'm not allowed to share :( ).

IMO, wavenet and pixelcnn/rnn demonstrate the power of convolutions as a general purpose AI and broke the reign of RNNs/LSTMs for many tasks.

(first paragraph referring to AlphaGo)

Meh. It's the old feud between fundamental research and applied engineering. You need both to keep moving forward. AlphaGo was a bit more in the applied camp.

Hi, sorry for the bother but the terms "RL^2" and "learning to reinforcement learn" aren't very googleable - would you mind sharing a link to the paper you're discussing?

Actually, googling 'learning to reinforcement learn' links to this paper - https://arxiv.org/abs/1611.05763

AlphaGo was a serious milestone and message to the public that AI has broken another barrier.

http://www.nytimes.com/1997/07/29/science/to-test-a-powerful...

In 1997 this was the general thinking about it: ''It may be a hundred years before a computer beats humans at Go -- maybe even longer,'' said Dr. Piet Hut, an astrophysicist at the Institute for Advanced Study in Princeton

I agree it was an impressive feat.

I don't agree that it was clear it was as far off as that:

I believe that a world-champion-level Go machine can be built within 10 years, based on the same method of intensive analysis—brute force, basically—that Deep Blue employed for chess

-- Feng-Hsiung Hsu, 2007, http://spectrum.ieee.org/computing/software/cracking-go

No, these are innovative _techniques_ for designing and training networks. What you're describing are just good _applications_ that "look cool".

Any links?

Reply on news.ycombinator.com