I have an encrypted USB stick for things like this. It has a keyboard, so encryption is built into the device and it wipes after 10 tries (not ideal for back-up).