Hacker News Reader

Comment by jknz | Hacker News Reader

The next person that will beat alphaGo may not be a top go player.

In particular, I'm wondering if a computer scientist with access to the alphaGo source code and all the weights of the network could trick alphaGo in order to win games automatically (cf. the papers that show a neural net can be tricked to classify a plane as any other class).

If a human with the knowledge of the source code and the weights can do this, it is scary. Imagine a similar algorithm runs your car. An attacker that knows the source code and the weights may trick the algorithm to send your car in a wall!

View on HN

You cannot do something unpredictable in a game with strict rules and borders. You had to trick this algorithm the billion times it trained for this game.

I had this conversation earlier with a friend wondering if any of Sedol's Korean pro buddies have noticed any systematic biases that could be exploited. I think it would be possible to make the neural net relatively useless by playing strange sequences that hack the weights, but you're still left with a monte-carlo-tree-search bot which alone depending on its implementation is between 2d and 6d amateur (so on the far weaker side of the professional scale). Whether you could make those strange sequences to trick the neural net while also not dying to MCTS, I'm skeptical.

I'm kind of hoping for an unconventional opening in game #3. Come on tengen or 5,5... Additionally I think that might be one way of weakening the bot in that it will find its net for suggesting candidate moves less useful and lean more on unguided MCTS, but that's just a wild guess.

I'm fairly certain that among the many thousands of high-level games from KGS AlphaGo was trained on, several opened tengen and 5,5.

Sure, almost any HnG fan will play them at least once. ;) But there's so few of them in general I would think any training data derived from them would be low-value.

I think this is less relevant

Tricking the NN works in a no noise situation, also an image has many more parameters than a Go board

Just let it play against itself. certainly one side will lose.

Though I wonder if something emerges. For example, black always wins, or, black always wins from a certain opening position.

Reply on news.ycombinator.com