If you're interested in these topics, the Sutton & Barto RL book is excellent, and Chapter 2 has a lot of discussion on exploration vs. exploitation strategies