One argument against local fine-tuning was that by the time you were done training your finetune of model N, model N+1 was out and it performed your finetune out of the box. That kinda stopped being the case last year though.