Comment by pimartin | Hacker News Reader

Comment by pimartin | Hacker News Reader

I think one of the things that you are missing out on big time with your existing SOC 2 report is the design of controls that are specific to your business and shows your customers the steps that you take to protect their data.

It sounds like you are doing amazing things from a security perspective, but that those are not included in your report, which means that your customers are missing out on. As you mentioned, sure you can put it in a white paper, but who is to say that you are actually doing it? That's the point of the SOC 2 report.

My hunch on this comes from the fact that you mentioned that SOC 2 comes from answering a giant spreadsheet from your auditors. It doesn't really, however that is how a lot of auditing firms get their clients to become compliant with SOC 2. They need something, and a spreadsheet is a good place to start.

You're right, some businesses won't need AV, or Background checks, but in your case if it makes sense to have a control that mentions "The company only leverages memory-safe languages", you should be able to, and even explain it further in your Section 3. The key is to design controls that address the SOC 2 criteria but also are 100% reflective of how you are operating your business. It sounds to me like you got to SOC 2, but that you could get a lot more out of it. This is unfortunately super common.

Source: I help SaaS companies with SOC 2 and I have been for 7 years now. Currently helping companies in unusual spaces such as crypto exchanges and making sure that all the amazing work they put in security is reflected in their report. I'd love to chat more in depth if you are interested, I am very passionate about this and I've worked with a lot of companies you probably use one way or another in your stack, or that are in a similar space as yours. Don't hesitate to reach out and good luck!

There's no formal audit I'm aware of that I would trust to capture and provide real assurance of the claims we'll make in our security practice writeup. I think it's more honest to just say directly that you're going to have to take us at our word on some of this stuff, that we're doing what we say we do. SOC2 does a great job of assuring that you don't have to trust us that we're terminating access when employees leave, and that we're conducting frequent access review meetings. It can't --- cannot --- do a good job of ensuring that we build secure software.

This is correct! Ultimately everything comes down to trust, there's only so much verification available. I've encountered companies who have SOC 2 while they blatantly do not adhere to their policies consistently. All SOC 2 demonstrates is that you wrote some words down and an auditor couldn't catch you in a lie after a few spot checks. That's it!

Even security questionnaires are practically unenforceable. If a vendor lies your only practical recourse is to avoid them.

I would much rather companies like Fly spend their time building and writing about real problems, including security, than figuring out how to abuse a SOC 2 report to demonstrate they're smarter than the average bear.

I hear you, and I think it lines up pretty well with my point below, everyone is going to have a different opinion about what they require to create that trust. To some, it might be a SOC 2 report, and to others it's having an understanding of the technical work that is being done behind the scene through whitepapers, meeting in person at conferences etc.

It is unfortunate that the SOC 2 process has become so mainstream because to your point (and I agree) there are a lot of weak audits. However, I feel like if you are putting in the effort of taking extra steps to be a better company and treat your customer data better, it is worth putting those controls in the SOC 2 report so that readers can know about it. Especially if you work with a recognized auditing firm. It doesn't mean that it is absolutely fault-proof, but it helps create trust, which is what it's all about.

On that note about trust, it can also go either way, as you've mentioned some SOC 2 reports will do the opposite of creating trust and will only result in more doubt and questions.

I don't deny that there are certainly companies that act in bad faith (say one thing in their SOC 2, but do another), but I don't consider it to be a fault of the SOC 2 process. Just bad companies. I wouldn't be surprised said companies would take shortcuts in other places aside from SOC 2.

I don't understand why taking the time to do SOC 2 right will take time away from the "real problems." Perhaps things like asset/vendor management, access control, and maintaining an efficient security organization aren't real problems for any organization. I'm reminded of that Futurama quote "When you do things right, people won’t be sure you’ve done anything at all." Unfortunately, just as you've encountered companies that lie on their SOC 2, I've encountered companies that have strong security engineering practices, but fails at basic organization security.

Thanks for taking the time to respond.

Ultimately companies become compliant with SOC 2 for one reason: Sales. It's the one department that wants it and the one department that plays no role in it.

One of the advantages of SOC 2 in my opinion is that it helps companies tell a narrative to their clients about how they operate. I understand your perspective of directly telling your customers how you are operating, and this might work for a lot of them, but some others will want to see it in the report. It also works to your advantage to only send one document at first rather than too many, because too many can open to many questions which will also slow down the sales process.

At the end of the day, since you have gone through the process once and will have to keep doing it for a while with the type II, I'd suggest keeping in touch with your sales team. Ask them what kind of questions they constantly get during calls with prospects. A lot of time it is very valuable to add controls around those questions. Anything you can do to reduce the procurement time leads to more sales, more business, and less engineering time spent answering security questionnaires that have become unbearable.

Unfortunately, there isn't a single standard that answers all questions that your sales team can provide, which is why I think SOC 2's flexibility is pretty awesome at helping close deals. The framework doesn't tell you what to do, but rather asks you how you address risks, which gives you a ton of flexibility.

Some companies can get away with very little and still flourish, others will need much stronger controls to satisfy their customers, and that's what it comes down to.

I've never considered SOC 2 as a way to ensure anything, but rather to educate the reader about how you are operating and have an accredited firm validate that you are indeed doing those things (to an extent with population & samples).

There are many ways to describe the steps that you take to ensure that you are building secure software, and I think you should get the credit for it, by having those important controls in your report, and help you stand out from the competition.

I do not in fact think we would benefit by having everything we want to say about our security mediated through AICPA auditors, even the excellent ones we work with. We'll write our own security documentation for prospects that care about how our security engineering works as well as whether we're SOC2-certified.

Bear in mind you're talking to someone who had a non-ironic debate about whether we oughtn't just do repeated Type I audits, year after year.

You sound super interesting and I’m sure plenty of people would be interested in chatting (including me!), so please think about putting your email in your profile!

Hey thanks! You bet, I just added it to my profile, thanks for making the suggestion. Feel free to email me, always happy to chat.

I have personal experience working with pimartin. If you're looking for a reference, they really know what they're talking about for SOC 2. They helped me get SOC 2 at the company I co-founded, ProcedureFlow where I'm VP of Engineering.

My main concern was this: we are a growing company and I didn't want to bolt on "some corporate SOC 2 thing" just to make us seem more secure. Honestly, my attitude was similar to the OP.

Sales were getting blocked and delayed by lack of SOC 2 but also having to fill out security questionnaires for every customer. I found pimartin and they really showed us how SOC 2 is customizable and isn't black and white like most people think. SOC 2 is not prescriptive about how you do things. He also helped us find an auditor that understands our business and made the process very easy for us.

When our prospective customers now do their IT/Security reviews, we pass with flying colors because of the changes that have been made to our organization and the big attitude shift we had about it. SOC 2 is not a burden in our company.

Happy to talk more about our experience with pimartin and doing SOC 2 "right"!

Reply on news.ycombinator.com