Yeah I think it would be a little overkill. The unique email addresses are almost impossible to guess so you just need to make sure you don't share them with anyone.
If the barrier is technical abilities, trying to set up public/private keys and/or signing the emails could hinder adoption.
Cool thing, man.