If, by "the actual board", you mean the HSM -- that hardly matters. If an attacker can seize control of the application processor over JTAG, they can command the HSM to do whatever they want.
The accelerometer and temperature sensors / settings could make it a little more difficult, since the heat sink is directly above the processor and one would need to get through it to gain access to the pins.
But it could be possible with some creative solutions. Probably not if all tamper protections are active though.