We also have code like that. For example, that's the explicit goal of the Spinning Up repo: https://github.com/openai/spinningup/blob/master/spinup/algo...

In practice, it's much harder to use that code, and we tend not to consume code like that internally. There's a real tradeoff!