There is nothing in theory preventing you from training while keeping a subset of weights locked.
How do you ensure that subset of the weights will continue to be accessible through the others?
I don't understand your question.
Assuming for simplicity that you're talking about locking layers. Each layer influences the input to the next layer, so locking a subset of layers doesn't necessarily mean that you'll continue to see the same results in the output. Not sure if you meant something more granular or otherwise, but I don't see how it's possible to simply freeze a part of a model in order to retain a particular capability.
I meant taking all the weights that are part of the cpu and locking those, and only those.
Ah wait, you mean you'd intend the other layers to be able to evolve to use the CPU internally, so all you need is to keep the CPU functional somewhere in the model? That may work, dunno.
Not a holistic answer but one example: There are specific fine-tuning techniques like Orthogonal Subspace Finetuning (https://huggingface.co/docs/peft/package_reference/osf) that target 'unused' parts of the weights so you can keep highly informative parts of the weights as-is.