The non-halting property of this setup is caused by the physical arrangements/movements of particles and the laws of nature. Fermat-theorem is an abstraction having no real causal power.
I would say, that for us the proof of Fermat-theorem is the right abstraction level to explain the general behavior of the machine: The machine does not halt because Fermat theorem is true
Similarly "punishment causes less crime" may be the right level of abstraction if the correlation is there, even if it can't be reduced to physical terms. And "Responsibility for action due to free will" may be worse abstraction if agent causation[1] is impossible (I am not saying it is)
So even if increasing/decreasing punishments is determined by low level laws of nature, we still can use the higher level explanation if it suits us better.