The public key is calculated from the private key, so it all still works, it's just a public key for a private key of 4.