(I don't know how the model is accessed - are users of mainline GPT-3 given a .pb and a stack of NDAs, or do they have to access it through access-controlled API?)
Wherever data is desired by many but held by a few, a pirate crew inevitably emerges.
> Earlier this year, researchers at NVIDIA announced MegatronLM, a massive transformer model with 8.3 billion parameters (24 times larger than BERT)
> The parameters alone weigh in at just over 33 GB on disk. Training the final model took 512 V100 GPUs running continuously for 9.2 days.
Running this model on a "regular" machine at some useful rate is probably not possible at this time.