Government spec motherboards shouldn't even have usb headers.
Sure it's possible to program a mouse or keyboard to inject code but way way way more difficult than a thumb drive or internet connected PC.
The first military to try to order such custom, extremely expensive (no alternative market for it!) gear will be flogged and quartered alive by the press. For spending 10 or 100 times as much as CotS hardware would cost.
Good luck finding a soldier brave enough to risk his career and family's income for mere USB ports.
The only hangup I see with that is that most boards don't have PS/2 anymore and you might need to connect a keyboard/mouse. To solve that you might need to epoxy a wireless transmitter into one of the ports and pair it with mice/keyboards as needed. But for situations where wireless devices are not allowable, I can't see any cheap solution short of soldering the USB wires to the motherboard. Damn USB for being so... universal.
Or just throw some software at it so that a very, very loud alarm goes off if a new drive is mounted or detected, search people for USB devices before they enter, and keep regular backups (if you make it impossible to get data out undetected, the only thing you could do with software on a USB drive is corrupt or delete data).