Use a VPN. Make that the only point of entry, and whenever somebody departs, you only have to revoke access to the VPN. Here's a decent OpenVPN guide: http://code.mixpanel.com/openvpn-in-the-rackspace-cloud/