Ask HN: how do you manage encrypted email? | Hacker News Reader

Ask HN: how do you manage encrypted email?

8 pointsrwl11 comments

I've recently (re-)decided to make an effort to use PGP, and to convince others to use it too. (My effort to do so: http://www.ocf.berkeley.edu/~rwl/encryption.html, linked from my .signature.) But I've run into a couple of problems fairly quickly. If you use PGP regularly, how do you solve them?

1) Reading encrypted mail that I sent. If I need to remind myself what I said to someone, or recover an attachment, etc., I can't, because the only copy of my message is encrypted with the recipient's public key. I could work around this by Bcc'ing myself on every message, but that would have the mildly annoying effect of duplicating all my outgoing messages; every time I were to look for a message I sent to Mr. X, I'd get two results, and I'd have to figure out which one was encrypted with my key to read it.

2) Search. The more serious issue is that I can't search encrypted email, whether I sent it or received it. It is conceivably possible to search mail encrypted with my public key by decrypting it before running the search (though not encrypted mail that I sent, pending a good solution to problem 1). However, that seems like it would be extremely slow in practice, and I am not aware of any software that would make this simple or practical.

I am currently using Mutt as a client for a local Maildir, with offlineimap and notmuch to download and index my mail, but I am willing to switch to a different setup if there is one out there that solves these problems. What's frustrating is that both problems seem pretty obvious, and solutions are conceivable, but search results for practical solutions elude me.

I don't want to recommend to others that they use PGP if it means they cannot reasonably search their mail archive and read their sent messages. What should I tell them to do?

Regarding point #1, on OS X I manage my encrypted mail via Mail.app and GPGTools, which builds support into Mail.app. This automatically encrypts messages with the public keys of both the recipient(s) and yourself, allowing you to read your own sent messages.

Thanks. How do you deal with the multiple copies problem? (Or is it all transparently handled by Mail.app?) Do you just not save copies of messages encrypted with someone else's key? keep your version in a special folder? etc.

GPGTools via Mail.app encrypts a single message with both keys rather than saving two copies with each encrypted to one key. When doing this via the command line interface you just need to specify each recipient when encrypting.

http://stackoverflow.com/questions/597188/encryption-with-mu...

What's the go to solution for doing the same on a Linux or OpenBSD box? I've been thinking of making the switch to some sort of *nix from OS X and I'd like to start on day 1 with 100% security practices. Moving to a new machine seems like the perfect way to start implementing everything in my life securely from the ground up.

I've never used Enigmail with Thunderbird, but it's the commonly recommended GUI implementation for Linux. Enigmail has a setting in the preferences to "Add my own key to the recipients list", which will let you decrypt your own sent messages.

The advice I got on the debian-users list for this particular issue was: use the "encrypt-to" option in your ~/.gnupg/gpg.conf, e.g. for me:

  # encrypt to me as well
  encrypt-to CF6FA646

This adds your own key as one of the recipients whenever GPG encrypts a message.

> The more serious issue is that I can't search encrypted email, whether I sent it or received it.

The tools to do this simply don't currently exist, but GnuNet has developed an approach to convergent encryption of keywords which points to how one might implement this in an email system.

The first step to do this would be to run the email through an extractor. It might do things like extract keywords from the email text (perhaps all non-common words and their frequency), attachment file names, MIME types &c. This extractor would then encrypt or hash these keywords and add an entry into the index for each keyword/email ID pair.

As a simple example, let's take your first paragraph. It might reduce down to the following words/weights:

((convince 1) (couple 1) (decided 1) (effort 2) (encryption.html 1) (fairly 1) (http://www.ocf.berkeley.edu/ 1) (http://www.ocf.berkeley.edu/~rwl/ 1) (http://www.ocf.berkeley.edu/~rwl/encryption.html 1) (linked 1) (make 1) (others 1) (pgp 2) (problems 1) (quickly 1) (recently 1) (regularly 1) (run 1) (signature 1) (solve 1) (too 1) (use 3))

You might then hash each element with your keyword hashing key, to get the following:

(("bea916398ce5fafc4497d378fc4555d9" 1) ("c81dc4aa3b1ae95487f288b297f51a6b" 1) ("8b801b4290ce10372b400330c737858b" 1) ("80b3e9090ee572563e67ebdecdb9b936" 2) ("cdd4c4efda80451e4dd1ff340dd2ed17" 1) ("9d4bcdc88159f8d830384f1696c32c19" 1) ("cb7baeb2e45e5e9d69fcf37c4df25669" 1) ("707f687e23bbbce127970361fd6bda10" 1) ("e6656976f2c6944f7aefd16b5786d6cd" 1) ("df9c284344194e1887e56478ec655dda" 1) ("ec8e0c526c105fc8ebb1ebdf293cf717" 1) ("f2d031d514795444be32f18690f6961f" 1) ("270273a76f0d974c2a06f96d40e2430a" 2) ("46bd41b18d2de638bbd3c126cb4ed716" 1) ("d96c629e01788ed763529620f308bd0c" 1) ("2d5fe4b610334160f178a1977b7d1db1" 1) ("211d0c274e8fb59bbcfe6e74241c317a" 1) ("ae5360545c4e008ed683df468136a8b0" 1) ("d6a5babaa2da167ef4548e7c34b49f40" 1) ("adcf3cd322f7ce16ad015f4725ddbda8" 1) ("e7e89d9c445263f9835c483aabf4f453" 1) ("84f0d11faa98a86f9624df86bd71901d" 3))

And then you'd store your document ID (in this case, the paragraph ID) and the weight in the index under each keyword.

To search for "use PGP", you'd extract the keywords "use" and "pgp", then hash them with your hash key to produce "84f0d11faa98a86f9624df86bd71901d" and "270273a76f0d974c2a06f96d40e2430a", look those up in your index, see that paragraph ID and the weights 3 and 1, retrieve the paragraph, decrypt it with your decryption key and you're in business.

Interestingly, this same architecture works securely if the index lives on another system, so long as the keyword hashing is executed on your system and your hash key is itself secure. Thus, there's no good reason that Google couldn't offer keyword-based searching on messages that only you can decrypt—no reason except that they do not wish to.

I would hope that such a security system also generates noise in the keyword index so that there is plausible deniability built in just like there is with TrueCrypt and fake/hidden partitions.

You surely could, although since the hashes are unpredictable to an attacker (due to the secret hashing key) I don't know why it'd matter, except possibly for traffic analysis.

Hmm, very interesting!

> The tools to do this simply don't currently exist, but GnuNet has developed an approach to convergent encryption of keywords which points to how one might implement this in an email system.

What you describe sounds a lot like "fully homomorphic encryption" (given my pretty amateur understanding of that term). Is the technique you describe one that could be layered on top of PGP? Or does it need an entirely different crypto scheme?

> What you describe sounds a lot like "fully homomorphic encryption" (given my pretty amateur understanding of that term).

Well, not really from a technical standpoint. Homomorphic encryption is about me encrypting 1 and 2, handing you #x8495 and #x7835, you executing (homomorphic-add #x8495 #x7835) to get #x1932 and handing it back to me, whereupon I'm able to decrypt it to get 3.

You can imagine why this could be cool: there's the potential for someone to be able to perform work on one's behalf without compromising one's privacy (imagine TurboTax working on encrypted data, with Intuit unable to know what one's finances are).

The technique I describe is different, but it does enable a similar end result: someone else can run an indexing/search engine on my behalf, and I can consult it to search my data without ever revealing that data to the engine-runner. The engine isn't really able to perform any calculations though.

> Is the technique you describe one that could be layered on top of PGP?

Well, it would work in parallel to PGP (or any other data-at-rest scheme). PGP would encrypt the plaintext, and this would be used to index the plaintext.

GNUnet uses this technique so that two different people can arrive at the same encryption for keyword searches (which of course has vulnerabilities, since one's adversary can perform those same searches); I use it so that the same person at two different times can index and search by keyword, and protect against an adversary through the use of a secret hash key.

Reply on news.ycombinator.com