Tokens, I think? There's an implication that there's a per-token cost, but no idea how they were architected before to even remotely understand how it fits. If they were running their own model on GPUs I guess I'd expect to see things in terms of token/sec, so I assume instead they were using some cloud learning model where they have to pay per token?

The blog post really suffers from "Written by someone who knows what they're talking about", could have done with a review before publication by someone who doesn't know the space. I go through that exercise any time I'm writing stuff for consumption outside of my service teams.

They say it cost 'thousands of dollars', so I infer that the 'millions' must be either raw characters or BPE tokens sent to the API.

Didn't you read the first paragraph... twice? Oh right, its not in there.

I believe it refers to number of tokens. So thats's $0.02 per 1000 tokens for OpenAI Davinci model.