Looking around I did just find a buildpack that tries to solve the problem. That doesn't really apply when using your service on my own servers though.
https://github.com/siassaj/heroku-buildpack-git-deploy-keys
I guess the bigger question is simply, are you going to limit your audience only to people already following best practices?
An SSL when transferring over these files, just based on the rest of the responses in this thread, would seem to make a lot of people feel better about the service.