> then deliver the page using that key and afterwards remove the other keys.

Per bullet 1, this seems self-defeating since at that point all that would need to happen is for the victim to forensically extract the private key from the compromised webserver. So long as there's one private key corresponding to a pinned public key, the malware's defeated if the private key can be recovered (assuming I'm understanding your suggestion correctly).

I say "forensically" since I'd presume any malware following this pattern would likely build in some protections to try and blow the private key in the event that someone tries to recover it.