Shameless self plug: if you're interested in Markov Decision Processes and related algorithms
, I am the author and maintainer of a C++ and Python library that implements lots of planning and reinforcement learning methods for them.
I hope to officially publish it later this year, but has already been used by a lot of people, in academia and out.
Feel free to play with it if you like!