fine-tuning original weights solves that, and any sane person would fine-tune for their task anyways to get better results
Of course, since the model is so large and general purpose already, I can’t assume the same fine-tuning techniques are used as for vastly smaller models, so maybe layers aren’t frozen at all.