This is also my thinking.. if someone compromises your VM that is responsible for retrieving wildcard certs from let's encrypt, then you're probably busted anyway. Such a machine would usually sit at the center of infrastructure, with limited need to be connected to from other machines.