Hacker News Reader

Comment by diafygi | Hacker News Reader

Daniel Roesler here, I feel really sorry for those who paid for Multi-AZ support on their RDS instances only to have all the availability zones go down. That would make me rage quite a bit.

View on HN

The only bump in service we had was ~30s of downtime when our multi-AZ RDS cut over to the failover instance. :) I think we lucked out a little bit...

Were you in us-east-1? That sounds amazingly lucky. If only Amazon would tell us how many AZs actually went down...

Does 1 AZ == 1 datacenter? It seems like it's possible 5 distinct "AZs" could have gone down while someone else escaped with all 4 of their AZs unscathed, since Amazon takes care to talk about availability zones rather than datacenters, and my AZs by definition aren't the same as your AZs.

AZ's are different data centers, but in the same region. I think that means that the buildings are close enough to have super low latency connections to each other. However, when a huge storm rolls through, it can wipe out all the AZs (which is basically what happened). AZs are mostly for guarding against floods, fires, and other localized disasters, not regional disasters that leave 2 million people without power.

As I understand it the separate AZs can be the same data center but in theory they are completely separated from each other in terms of network/power etc so an outage in one shouldn't effect the others.

The other catch that not many folks seem to realize is that 'your' us-east-1a is NOT the same as 'my' us-east-1a, although my 1a and 1b are guaranteed to be different AZs.

And I think you hit the nail on the head - 'regions' are for regional fault tolerance, while AZs are for within-a-region fault tolerance.

My multi-AZ RDS also failed over without a problem. The site was down for 5 minutes though, perhaps due to ELB problems.

Nice write-up. How did your users perceive the downtime? Mostly understanding, or was there a lot of anger?

Our entire team was making sure we'd give immediate response to any user who asked what was going on while we didn't have the down page up, so most people were pretty happy:

http://i.imgur.com/hCP44.png

This webpage is not available

Reply on news.ycombinator.com