Once you have access and either root or pass whatever extra authentication the access requires.
> storing the key encrypted on a drive where its only accessible using a passphrase
The extra failure mode here is that someone can copy the key and crack your password offline on their own time.
> using something that was touched and left a mark and some glue.
At that point a camera catching your password should also be a potential threat. But yeah, if that's something you're realistically worried about then it's not a great solution for you.