The Varnish authors have specifically stated they didn't want to introduce SSL into their process because of the added attack surface that adds. [0]
If you want SSL for Varnish, use something like Pound [1], stunnel [2], or HAProxy [3] to do the SSL termination and pass it off to Varnish...
I'm partial to Pound because it is lightweight and doesn't try to do any caching, it simply terminates the SSL and passes the request on.
[0] https://www.varnish-cache.org/docs/trunk/phk/ssl.html