I don't see how that's possible given they're likely training on multiple data centers' worth of GPUs scattered around the world. So you still need to get updated training data, software upgrades and everything else onto those machines. And then get the finished product out. And with iteration speeds in AI it likely needs to happen on a daily basis or multiple times a day.

“Completely air-gapping this infrastructure—that is, disconnecting it entirely from external networks—isn’t currently feasible, since we need external connections in order to run and monitor experiments”