OP refers to GPT-2. OpenAI has since then published both the code and the trained model.
It's getting harder to define 'OpenAI'. It is just another DeepMind, where they release the paper but never the model.
On the plus side, if their claims about how GPT3 was constructed are truthful it should be possible for outside parties to reproduce it-- though at considerable cost.
> We are still considering release of the larger models.
Is there something I'm missing?