Comment by dsr_ | Hacker News Reader

Comment by dsr_ | Hacker News Reader

Parent Full thread

At 3 AM on a Sunday, you discover a critical bug in a library provided by a third party.

Commercial support: send email. You get an automated message back with a ticket number. Around 6:30 AM you get a call from Delhi that satisfies the 4 hour response SLA, and the support analyst says that they will look into it. At 11 AM you get a request for a dump and certain logs, some of which you had already sent them...

Open source support: after getting zero google results, you look through the source code while running a debugger. It takes you two hours to find the place where it makes an assumption that a signed variable will never be negative, an hour to test your fix, and five minutes to send a patch to the mailing list. It is coming up on 6:30 AM, and you conference in the two people who need to approve emergency changes in order to get that approval. (This is a bank. You can't unilaterally make a production code change.) Everything is deployed before 8 AM.

Or, alternatively:

3 AM on a Sunday, you discover a critical bug in a library provided by a third party.

Commercial support: You call them. It takes an entire day of awkward back and forth emails and phone calls, but you eventually get an engineer to send you a a patched library for testing. It works.

Later, when auditors ask you about vulnerabilities, you can ensure them that you track $VENDORS supported long-term releases, and your contract has a security and support SLA.

Open source support: you find nothing on Google or Stack Overflow. You haven't looked at this source since your initial code review (you're a bank, you review all third-party code in use.) It's 3am, you're tired, and none of this is making sense. You've posted in the projects IRC channel but nobody's listening, and there's no good responses to your posts to the mailing list. You have to pull other team members out of bed, and together you get a reasonable test and patch implemented, which for some reason the maintainers aren't accepting quickly (if ever) and now you're running a local fork of your library. Hope your internal security auditing practices are solid, because there's no guarantee you didn't accidentally create a vulnerability during your pre-morning hackathon.

---

I don't love commercial software typically, but the support and auditing benefits are frequently the overriding factor when choosing for a regulated industry.

some good points there, but LOL about all 3rd party code being audited.

Most importantly, you can pass the blame to the third party organization rather than bringing that in house

We had to: not one line of code not audited, 3rd party or otherwise. Frontend and backend.

Wow, can I ask was it a big stack? And when you say audited, do you mean manual code review, static analysis or something else? How far down the stack did it go?

I'd be genuinely interested to hear, as it's something I see bandied around a lot but extremely rarely followed through on as code reviewing a modern stack is a huge proposition, and manual code review is very expensive/time consuming.

I have quite a bit of experience with commercial database support. You are missing a few steps.

Normally it goes for India for a few hours where they rattle off something that is in the knowledge base. Then when that doesn't work it goes to the next tier which is some support person in the US who does exactly the same thing. Then finally you end up with a support engineer who is finally technical and will usually do a screen share to walk through a solution. But if it's a bug you will have to wait for an actual product engineer who will finally start looking through code. I've seen this process take days.

And yet people are acting as though the big RDBMS vendors are somehow a good fit ? Crazy.

Your open source support case is 100% true if you know the language it is written in and the debugging tools. If not, 3am is not a good time to learn it. So for something at the heart of your business, choose a tech which has a community you trust behind it, whether that community is a longstanding FOSS community like Postgres, a company earning money through a closed-source embargo, or a well-developed internal team.

As another commentor said, it very much depends on how much you're paying for support. If you pay enough you can get a much better response than that.

Your open source case is also perhaps a touch optimistic, it implies you have technologists capable of debugging and patching every single part of your tech. stack.

As companies grow and need more software and also as the tech ages that gets less and less likely to be the case in my experience.

It all depends on how much you are willing to pay for that support.

From friends' experiences, Microsoft and IBM support can be very very good for the right price.

That said, I know at least some financial institutions use their fair share of open source tools. They're often best of breed.

while it may be true in some banking places, in others you would hit many walls with this kind of argumentation, or even bugfixing. many banks are very conservative in decisions, and IT department is no exception

Reply on news.ycombinator.com