The site was using one K80 GPU but it's slowing down significantly so I'm adding a second GPU. The servers run on Google Kubernetes Engine.
I'm also working on implementing that particular workflow with GPT-2/GKE/GPUs (I'm curious on your deployment strategy if you want to talk more).
You may want to use preemptible GPUs if you aren't already.
A quick guess - a standard server instance. It's hosted on what looks like a Digital Ocean IP, and Digital Ocean doesn't have GPU instances AFAIK; and generation is very slow, much slower than I'd expect from GPU. So this might be running on CPU, which would save a ton of money.
It was relatively speedy before it hit the front page.
(also as an aside, I do have my scalable/virtually free GPT-2 API on a CPU ready, should be released this week.)
I would be stoked to have some GPU resources from DO. I would like to play around with this kind of stuff on DO for a few days and then destroy the droplet. I have done it a few times with other OpenAI projects using CPU. It was pretty cool to see Ai play all the games in the gym. That is all, the end