I think the best thing you can do to speed up HTTPS is to move to HTTP/2. Check this out: https://www.httpvshttps.com/