Hacker News Reader

Comment by hacker_9 | Hacker News Reader

1. In a virtual environment the computer knows about all the objects in the scene. It's a whole other level to have the computer interpret the world through light alone.

2. If you meant that it learnt to play 2d games through watching pixels, it still didn't interpret any meaning, and it's still a huge leap to go to interpreting the real world.

3. The real world is noisy. You don't see it because the brain handles all that part for you, but even something like different lighting or a smoke cloud can throw a computer off completely.

View on HN

1, No, that's not how Deep mind works. The games are unsupervised and the computer knows nothing about the game.

2. Not at all. Haven't you seen the videos of computers walking around recognizing things? This is already here.

3. The real world is noisy, yes, but a factory is not. It will have a limited set of components to be recognized.

The training is most certainly supervised. The fact that people commonly draw a distinction between standard supervised learning and reinforcement learning isn't really relevant here (the distinction between a single datum and a sequence of observations), because the fact is the label is the score. If you read their DQN algorithm (or try to code an implementation, or look at any reinforcement learning algorithm, really) you would see that.

https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf: In Algorithm 1, see variables r and y_j.

Addendum: My broader implication is that if you misunderstand such a basic aspect of machine learning (and reinforcement learning), it makes me question whether we should lend credibility to your predictions about the field.

Edited to address reply.

Well, actually it's reinforcement learning and not supervised. Anyways, the point is that no knowledge of the game is required, just the score.

Knowledge that the score is the score, is knowledge of the game. Deep reinforcement learning uses the score as an objective measure for how well it's doing. That's is the definition supervised learning.

Reinforcement learning is not strictly what is considered supervised learning in ML, but it's very much in the same vein. And a supervised learning algorithm doesn't have any "knowledge" of the domain it's learning about either - it just adjusts its parameters based on training example and class/output pairs. RL attempts to find the best actions to take to ultimately maximize a measure cumulative reward, i.e. a signal which provides an objective measure of performance (much like the class or target output of a training example used in supervised learning).

RL is definitely not unsupervised learning, which in contrast, attempts to find some structure in unlabeled data.

Does it enjoy playing the game?

Reply on news.ycombinator.com