A hacker who got into your laptop can’t simply copy your id_rsa file to get further access to all your servers, it’s physically stored in the usb dongle with no way to get it out and requires physically touching it each time key is used.
Not being able to extract SSH keys helps very little.