ddpg() takes 17 parameters and is over 200 lines long. I'm very far from being a domain expert, but having worked in other complex domains, I'm pretty confident this can be redesigned such that it's both more maintainable and more pleasant to use.
Feel free to reach out by email, jachiam at openai.
It may be an indication that the whole thing needs refactoring though.