Then you have stable baselines which implements popular reinforcement learning algorithms to solve these gym problems: https://stable-baselines3.readthedocs.io/en/master/
Shamless plug: I've built a series of games where you solve puzzles (2048) / toy problems (MDP) like the lunar lander using various AI and ML algorithms.
You can check it out here: https://ai-simulator.com/