> training on multiple data centers' worth of GPUs scattered around the world
...so stop scattering things across many data centers.
> iteration speeds
Which can be slowed down for the sake of not breaking the law.
> it likely needs to happen on a daily basis or multiple times a day.
Keep it inside the airgap until you are ready to move the model into production. What's the problem here?