Hacker News Reader

Comment by tptacek | Hacker News Reader

There's no formal audit I'm aware of that I would trust to capture and provide real assurance of the claims we'll make in our security practice writeup. I think it's more honest to just say directly that you're going to have to take us at our word on some of this stuff, that we're doing what we say we do. SOC2 does a great job of assuring that you don't have to trust us that we're terminating access when employees leave, and that we're conducting frequent access review meetings. It can't --- cannot --- do a good job of ensuring that we build secure software.

View on HN

This is correct! Ultimately everything comes down to trust, there's only so much verification available. I've encountered companies who have SOC 2 while they blatantly do not adhere to their policies consistently. All SOC 2 demonstrates is that you wrote some words down and an auditor couldn't catch you in a lie after a few spot checks. That's it!

Even security questionnaires are practically unenforceable. If a vendor lies your only practical recourse is to avoid them.

I would much rather companies like Fly spend their time building and writing about real problems, including security, than figuring out how to abuse a SOC 2 report to demonstrate they're smarter than the average bear.

I hear you, and I think it lines up pretty well with my point below, everyone is going to have a different opinion about what they require to create that trust. To some, it might be a SOC 2 report, and to others it's having an understanding of the technical work that is being done behind the scene through whitepapers, meeting in person at conferences etc.

It is unfortunate that the SOC 2 process has become so mainstream because to your point (and I agree) there are a lot of weak audits. However, I feel like if you are putting in the effort of taking extra steps to be a better company and treat your customer data better, it is worth putting those controls in the SOC 2 report so that readers can know about it. Especially if you work with a recognized auditing firm. It doesn't mean that it is absolutely fault-proof, but it helps create trust, which is what it's all about.

On that note about trust, it can also go either way, as you've mentioned some SOC 2 reports will do the opposite of creating trust and will only result in more doubt and questions.

I don't deny that there are certainly companies that act in bad faith (say one thing in their SOC 2, but do another), but I don't consider it to be a fault of the SOC 2 process. Just bad companies. I wouldn't be surprised said companies would take shortcuts in other places aside from SOC 2.

I don't understand why taking the time to do SOC 2 right will take time away from the "real problems." Perhaps things like asset/vendor management, access control, and maintaining an efficient security organization aren't real problems for any organization. I'm reminded of that Futurama quote "When you do things right, people won’t be sure you’ve done anything at all." Unfortunately, just as you've encountered companies that lie on their SOC 2, I've encountered companies that have strong security engineering practices, but fails at basic organization security.

Thanks for taking the time to respond.

Ultimately companies become compliant with SOC 2 for one reason: Sales. It's the one department that wants it and the one department that plays no role in it.

One of the advantages of SOC 2 in my opinion is that it helps companies tell a narrative to their clients about how they operate. I understand your perspective of directly telling your customers how you are operating, and this might work for a lot of them, but some others will want to see it in the report. It also works to your advantage to only send one document at first rather than too many, because too many can open to many questions which will also slow down the sales process.

At the end of the day, since you have gone through the process once and will have to keep doing it for a while with the type II, I'd suggest keeping in touch with your sales team. Ask them what kind of questions they constantly get during calls with prospects. A lot of time it is very valuable to add controls around those questions. Anything you can do to reduce the procurement time leads to more sales, more business, and less engineering time spent answering security questionnaires that have become unbearable.

Unfortunately, there isn't a single standard that answers all questions that your sales team can provide, which is why I think SOC 2's flexibility is pretty awesome at helping close deals. The framework doesn't tell you what to do, but rather asks you how you address risks, which gives you a ton of flexibility.

Some companies can get away with very little and still flourish, others will need much stronger controls to satisfy their customers, and that's what it comes down to.

I've never considered SOC 2 as a way to ensure anything, but rather to educate the reader about how you are operating and have an accredited firm validate that you are indeed doing those things (to an extent with population & samples).

There are many ways to describe the steps that you take to ensure that you are building secure software, and I think you should get the credit for it, by having those important controls in your report, and help you stand out from the competition.

I do not in fact think we would benefit by having everything we want to say about our security mediated through AICPA auditors, even the excellent ones we work with. We'll write our own security documentation for prospects that care about how our security engineering works as well as whether we're SOC2-certified.

Bear in mind you're talking to someone who had a non-ironic debate about whether we oughtn't just do repeated Type I audits, year after year.

Reply on news.ycombinator.com