Hacker News Reader

Comment by dweekly | Hacker News Reader

> "DeepMind has found a way around this by employing vast amounts of computer power. AlphaGo takes thousands of years of human game-playing time to learn anything."

It seems the author may not have been familiar with AlphaGo Zero, which used substantially less processing power. https://deepmind.com/blog/alphago-zero-learning-scratch/

View on HN

AlphaGo Zero uses substantially less power to play, but it used astronomically more compute power to train & learn. According to OpenAI, AlphaGo Zero used more than 1,000 Petaflop/s-day, or about 4x higher than Alpha Zero and 100x more than Dota 1v1 or 10,000x more than VGG/ResNet. [1] The combination of better algorithms + more efficient hardware has significantly reduced the energy waste of that additional compute power.

[1] https://openai.com/blog/ai-and-compute/

Less power doesn't necessarily mean fewer games. According to the paper on AlphaGo Zero, they trained it on ~4.9 million games.

> Over the course of training, 4.9 million games of self-play were generated, using 1,600 simulations for each MCTS, which corresponds to approximately 0.4 s thinking time per move.

Assuming a go game takes 30 minutes on average, and you are never sleeping, resting, etc, you can do approx 18k games per year. In order to reach 4.9 million games you'd have to play for approx 280 years. So yeah, definitely not thousands of years :). Still, we are maybe one or two orders of magnitude away from the amount of games that humans need to play to become world class players.

That being said, the AlphaGo zero paper ends with the words:

> Humankind has accumulated Go knowledge from millions of games played over thousands of years, collectively distilled into patterns, proverbs and books. In the space of a few days, starting tabula rasa, AlphaGo Zero was able to rediscover much of this Go knowledge, as well as novel strategies that provide new insights into the oldest of games.

I doubt a human could learn to become even remotely competitive with only self-play within a human lifetime. Go has improved via a distributed effort, so we should try to estimate the number of go games played by humanity (as an upper bound).

Good point. I guess this ability to condense knowledge into language and pass it on has brought us where we are today. Genetically, we aren't that different from cavemen who lived tens of thousands of years ago.

> In order to reach 4.9 million games you'd have to play for approx 280 years.

Humans also benefit from millions of years of evolution which shaped our brain architecture in a specific way, and from a rich environment to learn from - nature and society. AG Zero was doing just self play.

The real question is, how do we train bots for environments which the responses cannot be well simulated, unlike in turn based games? They can’t play against themselves, then. They have to just sort of fit recorded data like backtesting, or do experiments that take time (“is this joke funny?”) or lots of time (“is this diet going to lead to an improvement in metric X?”)

How do we solve that?

Reply on news.ycombinator.com