Suppose then prevailing rates go up to 10%. Now you have incentive to take less than a million dollars if I pay it off now. You could take the million bucks and lend it back out at 10% now, but you're stuck with 5% with me.
The question is "how much incentive/how much less would you rationally take?"