Hacker News Reader

Comment by jurassic | Hacker News Reader

> Tape Gateway stores virtual tapes in Amazon S3, Amazon S3 Glacier Flexible Retrieval, and Amazon S3 Glacier Deep Archive, protected by 99.999999999% of durability.

That’s a lot of 9s.

View on HN

I hate it. Once the number gets so small beyond comprehension it’s hard to believe the validity. It reminds me of the Feynman’s statements during the Space Shuttle Challenger disaster.

> Feynman was disturbed by two aspects of this practice. First, NASA management assigned a probability of failure to each individual bolt, sometimes claiming a probability of 1 in 10^8, i.e. one in one hundred million. Feynman pointed out that it is impossible to calculate such a remote possibility with any scientific rigor.

https://en.wikipedia.org/wiki/Rogers_Commission_Report

Unless S3 can survive nuclear war or an asteroid hitting the earth, I don’t buy it.

There is rigor behind the claim. Check out this talk from a few years ago: https://www.youtube.com/watch?v=DzRyrvUF-C0

There's rigor within the assumptions. The problem is when the assumptions are wrong - most notably around failure independence. Data center fire? Accounted for. Simultaneous terrorist attack / asteroid (as per GP), / nuclear war on all data centers? Maybe not. Where does insider threat fit in on that spectrum? We don't know.

I'm personally fine with the claim as I understand the methodology and assumptions but it has to be interpreted carefully.

Yeah, the most depressing part of my job is when I have to calculate the SLO metrics for our directors. The first pass of stuff we control is always way within SLO. Then we have to subtract stuff where some third party messes up, or like the request doesn't even get to our service and it always puts us in the red.

One bit change in a sea of 100TB is quite a small percentage. Could work that way.

It means that 1 bit out of 12.5 GB is not durable. Not a lot if you ask me.

Think about it in terms of erasure encoding where you can have enough shards on enough hosts. That's how you get the durability.

Yeah but that's also incredibly naive. Inevitably there will be some un-calculated-for shortcuts that make all those copies not as failure-independent as they seem. If nothing else, what are the odds some random Amazon engineer makes a software rollout or hardware upgrade mistake and takes out a whole lot of copies in the process?

Does than mean that you should expect to lose 1 byte for every 100 GB that you store?

The statement is made at an object level typically for AWS

Reply on news.ycombinator.com