Yep, but running inference on it at any reasonable performance requires you to have all of it in GPU RAM - Ie. you need a cluster of ~100 high performance GPU's.
The largest version of GPT-3 is 175B parameters, which is ~350GB. I frequently use 8x and 10x RTX8000 boxes (and can access a 16x JBOG system as well), and the 8x system would have 384GB of VRAM. These sort of systems start off at only ~$60k.