Hacker News Reader

Comment by judofyr | Hacker News Reader

I think the developers explained the reason for this in a Reddit thread: Collectively a bunch of 1100 players are stronger than 1100. Imagine that a 1100 player will play one bad move for every two decent moves. However, different players miss different moves so the most picked move in each position will usually be a decent move.

View on HN

This makes perfect sense - but is a bit problematic given the intended goal of the project. They've built a bot that plays like an 'averaged group' of humans, not a human. Maybe they need a better way of sampling from the outputs.

This is always the problem with training from historical data only: you’ll become very good at being just as good as the sample group.

Ideally you just use the sample as the basis, and then let an AI engine play against itself for training, and/or participate in real world games, such as they did with AlphaGo and/or AlphaStar.

Seems analogous to the average faces photography project where the composite faces of a large number of men or women end up being more attractive than you'd imagine for an average person.

This only true if you select the most likely move instead of sampling from the probability distribution over possible moves. In the latter case there is no reason for there to be a wisdom of the crowd effect.

If you sample from the probability distribution you are modeling, there is no reason it shouldn't play like a 1100 player.

Well one reason is that it still won't be a 'single' player - what they've done here is like having a group of thousands of 1100 players vote on a move. What you're suggesting is to then pick a random player each move and go with them. There's no consistency, maybe on move 10 the player is blind to an attacking idea, but then on move 11 suddenly finds it...

I think they are saying, if your neural network was probabilistic and you thought there was a 90% chance of someone doing move A, but a 10% chance of move B, then you shouldn’t always get move A if it was human like - you would sometimes get move B.

I.e. most of the time if you leave your queen hanging and under threat your opponent will take it, but sometimes they just don’t see it. That’s the difference between playing a bot and a human a lot of the time - humans can get away with a serious blunder more often at low level play.

That's exactly what I'm saying - except more like the model is saying there's a 90% chance that a randomly chosen player at this level would make the move.

Yes, if you don't condition on the past moves then the distribution you're modeling is where you randomly pick a 1100 player to choose each move as you say. What I'm saying is that there will be no wisdom of the crowd effect.

> maybe on move 10 the player is blind to an attacking idea, but then on move 11 suddenly finds it...

What part of that is unrealistic? This happens constantly to human players, and not only at 1100...

Unintentional ensemble learning :)

Maybe. I think if you had asked me to predict the rating I would have guessed below 1100 though. Because of only predicting moves in isolation. I would expect that the moves would not form a coherent whole working together in a good way.

So it's an interesting result to me.

> Because of only predicting moves in isolation.

This actually may be the reason of higher ranking. What are the odds that a low ranked player will blunder a piece in a particular position? Quite low. But what are the odds that a low ranked player will blunder a piece in a game? They are quite high. So while this engine may predict most likely move, it can’t fake a likely game because it is too consistent.

I think GANs can be helpful to do something like this. One NN tries to make a man-like move and another one tries to guess whether the move was made by a human or the engine given the history of moves in a game.

I suppose the game at 1100 is really bad, such that it's mainly about avoiding obvious blunders, and not about having a sound long term strategy.

That's what a few machine learning people I talked too thought would happen. There are lots of examples (self driving cars being the big one) in machine learning where training on individual examples isn't enough. We were actually hoping for our models to be as strong as the humans they're trained on so we are underperforming our target in that way.

I'm the main developer for Maia chess

Have you thought about trying a GAN or an actor-critic approach? Did you find it infeasible?

Reply on news.ycombinator.com