I'm not an expert on this subject, but I have done a bit of gpt3 finetuning through their api:
I think it's clear that "fine tuning" with GPT is different from fine tuning something like Llama2, in that it probably isn't adjusting all the weights of the network, only a tiny subfragment of the network- Exactly how OpenAI accomplishes this is properietary.
The tradeoff is that OpenAI fine tuning is less expensive, but it is also less powerful than "real" fine tuning.