If all your endpoints are at least somewhat under your control (i.e. the thing doesn't need to be accessible via a web browser on a vanilla computer) you can reduce "exploits" to "exploits of my implementation of TLS/my VPN of choice" plus the bootstrap issue.
If the endpoints are not under your control to that extent, then I'd argue that "all the residential ISPs in US" is just as bad as "all of China/Russia" due to botnets.