Hacker News Reader

Comment by blazespin | Hacker News Reader

Yeah, I'm not so sure. All it takes is an intelligent agent that has trained in a VR environment accidentally released into a RL environment and do something crazy - like kill someone. That will be a taste of things to come.

Deep Mind is training computers to WIN in virtual environments right now. What if they train them against simulation of real world environments instead of video games? What if a computer discovers that blowing something up is a way to get a 'higher score'? What if they accidentally get pushed to production?

It's war games / bostrom's paper clip monstrosity. it can happen.

Arguably, it may have already happened in a flash crash. Create a simulation of a stock exchange, train a computer to win the market, it discovers that crashing the market to makes lots of money, gets pushed to production.

Instant global chaos (though maybe the AI owner gets richer). The future is now, folks..

View on HN

1. In a virtual environment the computer knows about all the objects in the scene. It's a whole other level to have the computer interpret the world through light alone.

2. If you meant that it learnt to play 2d games through watching pixels, it still didn't interpret any meaning, and it's still a huge leap to go to interpreting the real world.

3. The real world is noisy. You don't see it because the brain handles all that part for you, but even something like different lighting or a smoke cloud can throw a computer off completely.

1, No, that's not how Deep mind works. The games are unsupervised and the computer knows nothing about the game.

2. Not at all. Haven't you seen the videos of computers walking around recognizing things? This is already here.

3. The real world is noisy, yes, but a factory is not. It will have a limited set of components to be recognized.

The training is most certainly supervised. The fact that people commonly draw a distinction between standard supervised learning and reinforcement learning isn't really relevant here (the distinction between a single datum and a sequence of observations), because the fact is the label is the score. If you read their DQN algorithm (or try to code an implementation, or look at any reinforcement learning algorithm, really) you would see that.

https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf: In Algorithm 1, see variables r and y_j.

Addendum: My broader implication is that if you misunderstand such a basic aspect of machine learning (and reinforcement learning), it makes me question whether we should lend credibility to your predictions about the field.

Edited to address reply.

Well, actually it's reinforcement learning and not supervised. Anyways, the point is that no knowledge of the game is required, just the score.

Knowledge that the score is the score, is knowledge of the game. Deep reinforcement learning uses the score as an objective measure for how well it's doing. That's is the definition supervised learning.

Reinforcement learning is not strictly what is considered supervised learning in ML, but it's very much in the same vein. And a supervised learning algorithm doesn't have any "knowledge" of the domain it's learning about either - it just adjusts its parameters based on training example and class/output pairs. RL attempts to find the best actions to take to ultimately maximize a measure cumulative reward, i.e. a signal which provides an objective measure of performance (much like the class or target output of a training example used in supervised learning).

RL is definitely not unsupervised learning, which in contrast, attempts to find some structure in unlabeled data.

Does it enjoy playing the game?

It will be interesting to see how a robot walking around really works out. I mean, we know that when it comes to predicting reality, we're just terrible at it. 3 bodies interacting under gravitation outstrips our ability to predict. You want to account for the wind resistance and nonlinear behavior of everything around? Good luck! Chaos theory will eat you for lunch!

If an AI had decided that flash crashes were a way to get rich, you can guarantee that Goldman Sachs would be running one 24/7 right now. High frequency trading systems are actually quite constrained. Even if you hooked together all the worlds supercomputers and somehow got them into the datacenter of the stock exchange, you wouldn't be able to maintain the trading volume HFT systems do with a deep learning system.

Also, as far as I know, the successful videogame playing systems that accomplish some degree of 'long term planning' (meaning more than a few frames) all cheat. They actually "learn" by trying and then rolling the system back. In reality, you can't roll the system back. So that would be a fatal handicap to their methods.

Reply on news.ycombinator.com