Hacker News Reader

Comment by alexchamberlain | Hacker News Reader

There are 3 axes of risk: probability that something goes wrong, the impact of something going wrong and the time to remediation when something goes wrong.

You're arguing that on shoring QA would reduce the probability of something going wrong. I'm neither going to agree nor disagree.

However, I think the failure here is to mitigate the impact of something going wrong. Their rollout plan was fundamentally flawed - it shouldn't have taken out so many machines at the same time. It should have been rolled out in stages, with only 1 machine at most at any given customer receiving early versions.

It's best to assume a bug will get through 1 day or another, and spend some time mitigating the other axes too.

View on HN

My argument is they decided to cut cost on QA. It's very likely a higher paid QA team would of caught this.

A higher paid QA might of told management, hey this is a very high risk change. If we're going to roll this out let's limit it to reduce the numbers of people affected.

If you on shore your core development, but outsource all of your QA, I'm forced to assume you value QA less.

"If we're going to roll this out let's limit it to reduce the numbers of people affected." Ime this is something senior developers would themselves do - and not only for changes they deem "high risk", but also by default.

I say this because this case a data file was changed. Probably done thousands of times without an issue.

QA would have never said "we need a staged rollout for this". Developers and those who set the process should do it.

Senior SDETs can and should set the process for deploying software fixes.

But many companies don't view SDETs as equal partners in the developing process.

Anyway the entire world knows they cut corners here.

I am not sure if pay of QA team is truly a factor, but it is extremely likely if you off-shore QA you don't give them the absolute power to override anyone below CEO/CTO...

The flawed version was only up for about an hour and 18 minutes. In that time it was able to have the impact that it did

https://www.crowdstrike.com/blog/falcon-update-for-windows-h...

Reply on news.ycombinator.com