MDPs are the core holistic framework for decision under uncertainty, especially pomdps and dec-pomdps. That and the various forms of the Bellman equation are with your attention. RL is a model free MDP.