Hacker News Reader

Comment by thefreeman | Hacker News Reader

I don't know that they specifically said there was no domain-specific knowledge. iirc they said they didn't "teach it the rules of dota" but they also said the training involved "coaching". I interpret that to include showing the bot useful techniques (like creep blocking) which the AI then learned leads to higher win rates, etc.

View on HN

They also mentioned that in the beginning the bot figured out that the best way to win was to not play the game (aka hiding in it's own base). Then it started running around wildly, dying to enemy towers in the wrong lanes at the map. So it definitely took some nudging getting it to do something more than being AFK in base.

How did it figure that? There's pretty much no way to win if you're just staying in base.

Barring faction imbalance, there's a 50% chance to win by staying in base. However, I'm not sure how running around and dying could negatively impact your win percentage unless the other bot is also outside of the base.

I don't understand what you mean. If you stay in base, the opponent won't, and will quickly win the game.

This is during the training phase, the opponent will be a similarily trained AI.

Ah, I see, thanks.

You might (if timed correctly) steal aggro from your creep wave which will lead you to win the game if the other hero is AFK.

You might also pull aggro from their creep wave, which I think should also win you the game in most circumstances.

The idea is that by staying in base you don't die, while your opponent (which is still pretty bad) roams around and dies.

This highlights the importance of gradual adversarial training: if the opponent was perfect, there would simply be no way to win, and hence no signal on how to gradually improve to eventually find the optimal strategy.

Also shows a weakness in most training algorithms today that don't have, for example, the capability to watch their opponent closely and try to mimic it's actions in case he's better than you. Humans do this so quickly you can see Dendi already tried replicating the AI's strategy after 1 loss.

It doesn't does not need any nudging.

The idea is that first generations run around dying so the optimal response is to just stay put and let the opponent die. This now has basically 100% win rate so both of them do it, lowering it to 50%.

The other part is that there are random variations to the strategy it uses. This it experiments - they won't stay in base forever since that is giving you only 50% winrate (based on whose creeps would push tower first).

This random variation then means that the bot might go out, autoattack a creep wave and then come back to to base. Or anything similar to that.

This new strategy now has almost 100% win rate and the stay in base is no longer viable.

If it went out and died to a tower, it doesn't matter, it can just not use it and there will be period of base AFKing again, but it only takes one improvement (pushing a wave or hitting tower couple times) to make the base AFK obsolete and force new progress.

> They also mentioned that in the beginning the bot figured out that the best way to win was to not play the game (aka hiding in it's own base).

"A strange game. The only winning move is not to play."

- AI "Joshua" in "Wargames", 1983

I think "coaching" just means "curriculum learning" (which is basically as you've described).

But in the future coaching might indeed just be a human expert talking in natural language to teach it how to be better! https://www.youtube.com/watch?v=L_jGDV_5gPA&t=6

Not whole lot but some of this stuff can be seen in this video - https://www.youtube.com/watch?v=wpa5wyutpGc

I still can't quite tell if this behaviour was coached to the bot or it figured it out itself.

Reply on news.ycombinator.com