Hacker News Reader

Comment by cperciva | Hacker News Reader

One zone in US-East had a significant outage in April; and only instances using EBS were affected.

I distinctly remember the Amazon status page (which unfortunately seems to have very little history) claiming that two availability zones were affected during that outage: one was fixed within two hours, and the other remained offline for over a day (and if you were as unlucky as me, was not fully recovered for multiple days, not just 30 hours).

Regardless, you have missed the point of this thread: I have a bunch of servers on standard non-cloud providers (like 1&1), and I have seriously had /one/ outage in the last EIGHT YEARS. In that case, a hard drive (just mine in one server, not half of the known Internet's) failed, was replaced, and my server was back up in a few hours.

Actually some of the issues extended to the entire East region. In particular, for quite some time I couldn't create a new instance in any East zone. At all.

True, but "can't instantaneously spin up a new instance" just means that for a few hours the US-East EC2 region was providing the same amount of flexibility as dedicated servers.

The cheapest zone, yes, so presumably the most popular. And who doesn't use EBS?

You sound like you think I'm being unfair. Why? Plenty of sites were badly affected by that outage; it precipitated a lot of self-examination at some companies I know of. I don't think I am overstating the issues here.

The four zones in US-East have exactly the same prices.

And I don't use EBS.

You mean availability zone. Well I guess me, Reddit, Foursquare, and plenty of other sites just got lucky in the bad availability zone.

Ah yes, here's the classic AWS apologist pattern in full effect. You don't use EBS! Of course you don't, you would have to be some kind of friggin' idiot to use EBS. So what do you use for, say, MongoDB data files, that is different from morons like me who stupidly assumed they could/should use EBS?

Only fools would want a persistent filesystem for their OS! </sarcasm>

How is S3 not persistent...

Notice I said for the OS. From Wikipedia:

"An EC2 instance may be launched with a choice of two types of storage for its boot disk or "root device". The first option is a local "instance-store" disk as a root device (originally the only choice). The second option is to use an EBS volume as a root device.

Instance-store volumes are temporary storage, which survive rebooting an EC2 instance, but when the instance is terminated (e.g., by an API call, or due to a failure), this store is lost." (Emphasis mine)

Most choose the EBS type of instance now, because having the OS on a persistent storage device is more convenient for typical server scenarios (as you might imagine).

http://en.wikipedia.org/wiki/Amazon_Elastic_Compute_Cloud#Pe...

The only scenario I can see that would require hitting an EBS store often enough to matter would be if you put your database on it. Obviously your OS kernel is going to be in resident memory, so your argument is somewhat of a strawman.

It's the root device that's on EBS. On most AMIs with EBS root that's the entire OS i.e., /etc, /dev, /bin, /usr, etc. and not just the OS kernel.

And for the things that matter (i.e., not fake mount points) that will all mostly live in resident memory.

... and your point is?

We use EBS, and had machines in the availability zone that went down that were affected. Those machines were out for longer than a day, but we were back up within an hour because we had redundancies built in across other availability zones.

If you're doing anything that matters, you can't rely on a single zone/machine/whatever, no matter who your hosting provider is.

So you're saying we need a cloud of clouds?

Just a higher level of abstraction. Many people don't want to care about what zone they're running in, or really anything about the machine (or even virtual machine) they're running on; they just want to run their apps "somewhere" that persists through hardware problems. Maintaining such a thing isn't Amazon's charter, though; it's more the job of a cloud application hosting provider, like Heroku (who I'm surprised isn't multi-homing their apps on several clouds by now for just this reason.)

Sounds like a certain exotic financial instrument...

S3?

How would you run your database off a file storage service?

I believe Amazon SimpleDB is proxied over S3 storage systems.

SimpleDB latency is bad, but not that bad.

I doubt you'll be able to do much better with another scalable storage system (that's freely available).

Reply on news.ycombinator.com