Responsibly Bringing a new Cryptography Product to Market

Be careful. It is easy to pay for an audit from a firm that doesn't have a serious cryptography practice. You would reasonably think that any software security firm would be competent in evaluating crypto, or at least crypto basics like whether you're using a sane block cipher mode or failing to authenticate your ciphertext. But it turns out that the opposite is true: the overwhelming majority of firms, including some of the best, have no crypto literacy at all. (The best firms that don't do crypto will tell you this and refer you).

We've seen some horror stories. It's hard not to get a little irritated when you're the second firm to assess a target and own it up on day 2 with a trivial CBC padding oracle.

All this means is, when you're talking to a potential auditor, ask them hard questions about cryptography. Ask them to describe some of the crypto vulnerabilities they have found on projects. If they talk about "weak keys" or "bad ciphers", they're unserious.

Zooko's team at Least Authority is a serious crypto practice. Engaging Zooko was a good call!

$1000 per auditor/day is less than you'd pay to get someone to run Nessus on your network from a normal firm. Zooko did you an _enormous_ favor. For crypto work, a rate four times as high wouldn't be out of the ordinary.

I just posted a comment on the SpiderOak blog about how I was rather startled when I found out that they had waited until after we did the security audit before they informed us that they knew about bugs going in:

https://spideroak.com/blog/20140220090004-responsibly-bringi...

However, after I got over my surprise, I started thinking that this was a really good move on SpiderOak's part. If you hire a security auditor, it might be hard for you to tell whether you're getting value for your money. Leaving known bugs in the code and then observing whether the auditors find them is potentially a good way to overcome that.

Mind you: this will make life harder for we in the security auditing industry if this practice takes off. ☺

I was one of the auditors for Least Authority. Just wanted to add that this audit was really fun to do! If you have the necessary experience and are part of a good team, security auditing is a fantastic job. -- Daira Hopwood

Interesting read. I remember Crypton being announced almost a year ago [0], and for awhile I paid attention the GitHub repo, but it really felt like a dead product. Looking back at the commit history, I'm not so sure why I had this impression, as it seems pretty lively. I'm thinking that perhaps a lot of those commits were not pushed publicly until recently, but in either case it's good to see it active again.

On a bit of a tangent, regarding this post and the requirements (1-5) they posted, I really wish they would hurry up and finally bring this to their backup clients. It's for far too long been a joke in my mind that they promise "100% Private", "Zero-knowledge", "Secure" online backup, sync and sharing with a closed-source client. I went as far as to sign up with them and download the client, only to never install it because I realized I didn't believe a word they said. They promised (again) at the start of this year to open source their client [1]. I have actually followed the issue closely, and just last year one of the founders was admitting on reddit that it simply wasn't a priority. I don't see why not, as it makes their service complete snake oil, and I can't be the only one who refuses to use their service because of it. Anyway, my point is... perhaps from their perspective this was because of point #4 in the post for this submission (Have clear, well documented code).

[0] https://news.ycombinator.com/item?id=5283072

[1] https://spideroak.com/blog/20140110144419-spideroak-to-becom...

Edit: Here [3] is where they prioritize "growth and profit" over providing what they promise. The granularity on the visible submission date is not very informative, but if you look at the posts JSON, you can see the Unix time in the field created_utc. It was the fist of April, so a little less than a year ago.

[3] http://www.reddit.com/r/IAmA/comments/rmp9l/we_are_spideroak...

By the way, once we've established confidence in the code directly included in the project, we're hoping to also arrange audits for some of the popular crypto libraries involved such as SJCL.

It's interesting to see what's involved in a security audit and how much it costs. It's nice to see this kind of information being freely given.

Some programming efforts can be casual, some can not. Cryptography is is one that often requires not just rigor, but also expensive investment. The combination can be an obstacle to open source inroads. I'm part of SpiderOak, and proud of their investment in Crypton. It is expensive - like, not just the coding, but sponsoring these audits - but, I believe, extremely valuable.

That's great you found good auditors, and that will probably work out just fine.

Another idea is that real attackers are your only hope (https://news.ycombinator.com/item?id=7238750) maybe. Auditors are supposed to simulate real attackers, and maybe these serious auditors are identical to attackers except you don't pay them as much as they could get some other way like publishing your exploits on the black market. Speaking of market, Schneier said you really need a mass market to test things (from the above link).

Least Authority's blog post about the audit: https://leastauthority.com/blog/least_authority_performs_sec...

Glad to see real numbers and specifics about the audit being used in the post.

This is critical. If you don't think about this stuff, use the existing code.