The way it works, Circle pushes the containers built to Google Container Registry and there is a super simple tool I built (https://github.com/crufter/puller) pulls those images. This introduces a couple of minutes of delay but we find that acceptable.
I guess we are still exposed to the possibility of someone breaking into Circle, using the keys there to push to our GCR repos, but at least they won't have full access to our servers...