Hacker News Reader

Comment by emcq | Hacker News Reader

If the value/policy model is predictive with a dataset containing only amateur games, but fails to generalize to unseen data with professional games, that seems like a case of overfitting to a dataset only containing amateur games. In this case the expected value network may be different for amateur games than professional games.

Is there something I'm missing?

View on HN

Sorry, I'm being a little academic. Overfitting is when the model fits to noise or error. Overfitting is not synonymous with "inability to generalize beyond the train and test distribution."

For all we know AlphaGo has perfectly fit amateur games, but professional games are on a whole different level

All depends on how you define your data sets I suppose. IGS games include some professional games which, if it is the case that AG is perfectly trained on the amateur mode, were smoothed away.

Again, the only way to tell if overfitting specifically (and not other factors that are more likely) is the issue is performance on a held out test set.

That's the only empirical way, yes. We can also just talk about what it would mean in theory, though. In this case, we'd say that AlphaGo is well trained to the training data set sampling distribution but that may be far from the actual world game distribution.

That does not relate to overfitting. That problem is closer to "domain adaptation."

Unlikely. Some of AlphaGo's moves were lousy moves, even for an amateur.

You are implying overfitting is the only reason a model ever performs poorly. That is not true. Just because a model does poorly does not mean overfitting must be the issue.

The value/policy model includes a few hundred thousand amateur games, and a few hundred million games of self-play. Once AlphaGo beat Fan Hui those would have been games of self-play versus the equivalent of a professional. So overfitting is probably not a problem. I think it's a basic incentive mismatch - MCTS algorithms tend to like close games, whereas humans will try crazy moves when losing to throw off their opponent.

Neither of those are really evidence overfitting isn't a problem.

You tell whether overfitting is a problem by evaluating performance on a held-out test set.

Wouldn't a million self play games exacerbate overfitting by learning it's own play style which it initially learned from amateur games? I

Reply on news.ycombinator.com