Said adversaries could just use take images of the VMs (which they probably do anyway for backups) and extract the cert from them.

Unfortunately, it seems this topic has attracted the attention of many who keep insisting that it must be Let's Encrypt at fault here.