This is really the biggest challenge facing borrowers. The means of payment should really be locked in. Fortunately the servicer doesn’t always change too, but that is something that the law should probably protect.
I could imagine the law requiring the consent of the borrower to the transfer of servicer, or some similar mechanism.