You can encrypt them. I have no control over the idiots at my telecom who will reroute my number for anyone who provides the correct answer to some easy 'security questions'.
I see what you mean but if you encrypt them you will end up with an extra encryption key... that you'll need to store somewhere else.. otherwise, holding just an encryption passphrase is pointless as it's just the same as having two login passwords.