Yeah, there is a lot of UX work that could be done to address this. For example, a "HTTPS green lock" equivalent for ongoing email conversations.