Don’t store the private key on the local filesystem then. Use a USB stick or build and sign the modules on a separate machine. Depending on your threat model, a strong passphrase on the signing key might already go a long way.
I keep my signing key on my machine, but gpg-encrypted bound to a yubikey. Is that frictionless? No, certainly not. Does it provide perfect security? No, certainly not. A dedicated attacker can root my box and wait until I need to sign a module. Does it protect me from loading random kernel modules if I get hit by an automated attack? Most likely. Good enough for what I currently expect as threats.