Hacker News Reader

Comment by svalorzen | Hacker News Reader

Shameless self-promotion post: if you're interested in AI based on Markov, I maintain an open-source library here https://github.com/Svalorzen/AI-Toolbox. It can be used for general autonomous agents which need to act in response to environmental stimuli. I've used it to learn to play StarCraft and to control cameras to predict people's movements around buildings. Any comments and suggestions are welcome =)

View on HN

Ah, I was confused for a second—I'd thought Markov was a library, but you meant the markov assumption—the topics are actually loosely related and orthogonal. Your excellent looking library deals with reinforcement learning agents that model environment/agent interactions as a (PO)Markov Decision Process, where as the Alchemy library combines FOL with network representations of particular (satisfying certain markov properties) probability distributions to perform inference.

More pertinent to your post, Sutton's working on an updated RL book here: http://people.inf.elte.hu/lorincz/Files/RL_2006/SuttonBook.p...

If anyou have the time, Chapter 15 (pdf pg 273) of the above link is a fascinating read. In particular, TD-Gammon had already achieved impressive results using NNs in the early 90s; reaching world class levels in Backgammon with zero specialized knowledge.

> I've used it to learn to play StarCraft and to control cameras to predict people's movements around buildings.

If you've described / published either of these examples somewhere, you should submit them to HN!

I actually have a video [0] and code [1] of the first (which was a one-month project), while the second is my thesis which is going to be finished soon. Maybe I'll post them as a separate entry one day, but I always think these things are too simple and trivial to be interesting..

[0]: https://www.youtube.com/watch?v=TrKMBIR82Qw

[1]: https://github.com/Svalorzen/DESOLATOR

Loved watching the Dragoons dance around their opponents while taking almost no damage. They seemed to have slipped a few times where they could've doubled up on fire but just one was firing instead. I might have looked at it wrong though.

Shameless naive question: are these techniques suitable for time series models?

You mean data-fitting? Possibly (every problem can be modeled as an MDP) but it's definitely not the most efficient nor the most intuitive way to go about it, so I wouldn't recommend it.

Thank you for this, I will look into it :). I was actually going to tell these people to give me a github link or go away, haha. I don't need any kind of license page that looks like http://alchemy.cs.washington.edu/license.php?action=2, thank you very much!

By the way, you can talk to my AI guy if you have a recent Chrome/Chromium browser. It is actually an entire OS in a browser, and the AI guy is an app called Bertie. My current implementation of the OS is called "The Native Client Proving Ground", and can be found here: https://nacl-pg.appspot.com/desk. The OS should install in less than 2 seconds after clicking the link. Then you just open the Applications folder and click on Bertie's face. He will then introduce himself, and you can start talking to him.

Reply on news.ycombinator.com