ChatGPT uses a modified version of GPT3 (details are sparse, but here are some: https://beta.openai.com/docs/model-index-for-researchers). There’s a logbook associated with the training of OPT (similar size to GPT-3): https://github.com/facebookresearch/metaseq/blob/main/projec...

This article estimated the cost of training GPT3 to be over $4M: https://heits.digital/articles/gpt3-overview