If you are going to do this right, you need email encryption. In which case, you have a client-side certificate. In which case, you might as well just authenticate over SSL with that certificate.
But hey, why do something secure and built-in to every browser (even IE) when you can invent your own protocol instead? It's the Internet!
Also, any reason why this is an "Ask HN" and not a blog post? (Not saying it should be one or the other... just askin'.)