The Economics of AWS
aws.typepad.com
aws.typepad.com
Bandwidth is ridiculously expensive with them. You can get 3-4x cheaper per megabit going dedicated.
Servers are crazy expensive. Compare the most powerful machine they have vs something on 10tb / gigenet / theplanet for the price. You will definitely end up with a more powerful machine for half the price on either.
The only real advantage to using ec2 is the hourly billing. Make its perfect for overflow, but thats about it. I read about alot of startups that use ec2 for things like webservers or other servers that have 100% reliance.
Dont get me wrong. Some of Amazons offerings are great. The CDS and S3 can take the bullshit out of dealing with the complexitys of each and might save you a network engineer or 2. However, for EC2 there is no excuse. Its no more difficult to setup a normal dedicated server than it is a EC2 box.
/endrant
The way I look at it is if all these services were free, EC2 would win based upon features. So, the question is, does cost matter?
For us, a single engineer's salary costs more than our entire yearly EC2 bill. And, we're not even using cost saving measures like elasticity and reserved instances.
And, EC2 actually drives costs down in ways hosting does not. First, as you mentioned, it dramatically reduces the need for IT staff. (Though it doesn't eliminate it, of course.)
Second, and IMHO, more importantly, the constraints EC2 imposes on you forces you to build fault tolerant, shared-nothing systems. I'm sure many folks here have worked at companies with hosted servers to find machines that are chock full of random cron jobs, services, and so on. The instant provisioning capability of EC2 combined with the constraints makes it very hard for you to have these one off "god boxes" your system is hinging on. Beyond that, as things get too messy, you have the freedom to automate and shut-down and bring up fresh nodes instantly with no increase in cost. It's hard to me to estimate the amount of man-hours we've saved and disasters we've avoided due to this capability. It's allowed us to cleanly "refactor" our entire cluster over time since we can reprovision nodes on demand as we improve the services or architecture on those nodes.
I think this is the important part of the debate. Many people don't use these features, but with EC2 you are paying for them regardless. For example, you pay for the ability to pay by the hour (in terms of increased costs), even if you never plan on doing so. Yes, reserved instances can help, but they don't fully cover the cost difference.
So it really does come down to what features you need. If you do use all the features, I'd imagine that EC2 is a great deal, because everyone else is subsidizing your use of them. If you don't want any of the features, well, then you're subsidizing everyone else.
I think the "cost shouldn't matter" argument isn't relevant. If cost doesn't matter, it is hard to make a case for EC2 vs spending the money to develop your own infrastructure + hiring lots of people + vastly overprovisioning your own datacenters.
"the constraints EC2 imposes on you forces you to build fault tolerant, shared-nothing systems."
EC2 forces you to care immediately, not later when the shit hits the fan (which it always does, eventually).
S3, SDB, CDS, etc.. Yes, those make things easier. EC2 Does not.
Is this from experience? If so, please share it.
Bandwidth is ridiculously expensive with them. You can get 3-4x cheaper per megabit going dedicated.
Yes. But that does not normally matter. Bandwidth has become ridiculously cheap either way; a terabyte sets you back a measly hundred bucks on amazon. And if you're pushing more than a few dozen of them then most of it is very likely going over a CDN (or a set of separate dedicated servers) anyways.
Remember sites pushing 2-digit terabytes of dynamic html are very rare. And everything else can be trivially offloaded (e.g. to one of the isp's you advocate).
Servers are crazy expensive. Compare the most powerful machine they have vs something on 10tb / gigenet / theplanet for the price. You will definitely end up with a more powerful machine for half the price on either.
Well, I did and came back underwhelmed - as usual. Yes, you can go cheaper for a comparable spec, sometimes as much as 50 bucks. But again - that only matters when you're a small shop with 1-3 servers, looking to cut the penny.
Once your business grows out of that the hosting fees turn into a rounding error either way, and the advantages EC2 offers are nothing to sneeze at (ever looked what an solution equal to EBS costs elsewhere?).
Its no more difficult to setup a normal dedicated server than it is a EC2 box.
That's a red herring. Provisioning is automated in any reasonable sized deployment and you face the exact same problems with dedicated hardware that you have with amazon. Plus the hassles of having to babysit said hardware...
There is exactly one argument against amazon today and that is when you need hosts with lots of memory. RAM is expensive in the cloud, for obvious reasons - to a degree where stuffing your own hardware with 64G starts to look like a bargain. But the thing about this argument is: Very few applications really need that amount of RAM in a single box. In most cases the root cause for such a requirement is simply a flaw in your architecture.
I wouldnt say text sites pushing over a TB are rare. I have a few myself doing over that ( petitionspot.com does 2+ TB a month ) - That particular site runs on a 5520 QC with 6 gigs of ram and 10tb transfer for 200$ a month )
If I was to put that same site on EC2, I would be paying significantly more, as I doubt even the large instance could handle it. 1.2ghz opterons are not exactly speed demons.
For the sake of argument, lets say the Large Instance could handle it though.
Pricing at 34 cents a hour = 252.96 a month Bandwidth costs, 2TB = 340.00 a month
So we can either pay 592.96 for a less powerful machine, Or 200, for one that has 8TB more bandwidth to play with, and has roughly twice the processing power. In case you are feeling frisky, you can upgrade the ram to 12gb for 75$ a month. Still significantly cheaper.
https://orders.gigenet.com/index.php?order=true&form=235... Heres a link to the server I mention above.
That's where EC2 starts to make sense - not always, but often.
For your site, without knowing the details, I have no doubt that you made a sound decision. You should just be open to revisit that decision once it grows to a point where a few hundred bucks difference a month don't weigh as much anymore. Or rather, where the added flexibility makes that difference worthwhile. - That's all I was trying to say.
You also do not differentiate between traffic in and out ("in" is currently free fo EC2 and in any case cheaper than traffic out) - this may be significant depending on the kind of service you run (for most applications it won't be).
Similarly: if you make use of other Amazon Services (in particular S3 which you too seemed to like) this would count as external traffic from servers external to amazon but not from EC2. However, again, for most applications that would probably not change the overall picture.
But i do see one argument that can change the picture: if you use ec2 for overflow, testing and short time workloads and some other hosting (in-house?) for everything else, then this significantly complicates your infrastructure, i.e. you need take care of your in house servers and understand ec2. This may be significant enough to run even long running servers in the cloud.
That's not really a safe assumption. I work with several real-world, profitable sites whose monthly bandwidth usage range from 50Mbps to 2Gbps at 95th percentile, and they push enough custom content that CDNs are overly complex and still not priced competitively. Their respective business models would not be viable if they ran on Amazon. I just helped one of them acquire a pair of GigE uplinks, one of which they're paying $1,500/month for the other they're paying $3k/month for. Assuming they can see 800Mbps out of each link, that's $7 and $14/TB of xfe. This compared to Amazon's "measly $100" ..
2Gbps is indeed a different ballpark. For most sites I have seen in that realm it would be ~90% static content (i.e. media). If your site truly needs to hit logic for each of these requests then yes, a distributed solution probably won't do there.
Anyways, I didn't mean to say that EC2 is the be-all for everything. I just wanted to place a counterpoint against that all too common, lighthearted cost-bashing which too often is not backed up by figures that make sense the way yours do.
These kids (I'm 30 now, I can totally pull off the crotchety old man thing) drink the cloud kool-aid and think "great, we don't need to think about our infrastructure". They slap together something that almost works, but is a management nightmare. It's no different from the startups I was consulting for in the '90s, except it's a lot quicker to use a brute-force scaling approach now.
I like S3 for what I use it for: cheap tertiary archival for ~20TB worth of backups. Beyond that though, it's hard to find ways for the numbers to work for the pure cloud hosting sites. Especially for services that are more than just simple CRUD apps. None of the startups I'm currently incubating can do more than maybe some simplistic web hosting there, since they all involve "complex" infrastructure needs. (I double-quote complex, because in my world, BGP route optimization or IPv4 anycast is just a run of the mill daily problem).
Any decent dedicated provider (eg- theplanet.com ) Will email you your server details 5 minutes after you order the box, unless you are getting some weird equipment or off the bend OS.
The bottom line is you are paying 2, sometimes 3 times as much for that convenience of "instantly" powering it on. How often do you that? How often do most users do that?
If your business is at all like most, Your need to power on a new server instantly isnt mission critical. Even if it is, You should have your main servers running OFF of ec2. By all means, use ec2 or any other VPS to handle your overflow.
We have a decent sized EC2 architecture right now and EBS snapshotting + provisioning on demand means that we can get all of the benefits of multiply redundant hardware at a fraction of the cost.
- We never need to upgrade software in place. Software gets freshly installed on a new cluster that duplicates our existing cluster and gets a increasing portion of traffic. Failover is as simple as a proxy cutover. The freedom to do config changes or 3rd party software upgrades and be able to just swap instances in and out enables us to experiment often and safely with production changes.
- Slave gets corrupted or falls out of sync with the master? A machine suffers a degraded disk? Bad net interface? Don't fix, replace.
- We can spin up an entire test stack that precisely mirrors our prod stack to stress test with significant cash outlay. Since this cost is low, we can stress test often and test the effect of s/w and config changes on performance easily. Stress tests the H/W outlays associated with them are traditionally hugely expensive if you want to test exactly like you run.
- Significant database migrations can take place on a slave that is easily promoted, snapshotted and cloned without having to deal with any long term ping ponging between instances.
- Copies of databases can be instantly provisioned and destroyed for data analysis needs. If analysis needs peak at 5x in a week, no one has to wait. Run 5 copies and shut them all off when you're done.
These are just a few quick things that we do with regularity. And I'm sure there are lots more opportunities to exploit these capabilities than we are thinking of.
Also - in terms of cost - don't forget to compare against reserved instance pricing.
I use r1soft for redundancy and the costs for that are nominal. That sort of handles any hardware issues for me. I can get back up and running in about 10 minutes from a catastrophic failure.
All the stuff you mention below really doesnt matter to me, and I cant imagine it matters that much to many people. All these tests you mention could easily be done off site, or mirrored to run locally.
Still not worth paying 3-4x for these random "abilitys"
For things that matter- Bandwidth, Processing Power, Ram- Dedicated servers have EC2 Beat, hands down. More power, Less cost.
How often have you tried it?
OTOH, do you have API keys for your Dell salesman and your co-location facilities? I'd love to see those provisioning scripts!
This is why I pay for EC2. My dedicated boxes are on a yearly contract and when it comes to move it's a couple hours to move data, IP addresses, etc.
Hence the major problem with ec2- Cost. Yes, you can instantly scale up to a less powerful server than you would have got in the first place from a dedicated provider.
Also, You do know you can do things like upgrade the processor and ram on a dedicated, usually within a hour also. You just send in a ticket.
What server provider makes you do a contract?
That is simply false. No idea what kind of beef you have with amazon but please get your facts straight.
You seem to consequently ignore that with amazon you not only pay for a piece of hardware but also for the flexibility and infrastructure that comes with it.
Yes, your cheapo host might give you a gig more RAM and a few more megahertz for your dollar - but it doesn't have equivalents to EBS, S3, CloudFront, instant provisioning and all the other goodies that make EC2 worthwhile.
It will probably also not give you means to recover within minutes when your shiney, dedicated pizza box decides to blow a fuse.
Most servers sold from either provider come with 1-2TB of bandwidth. How much does 2TB of bandwidth cost on amazon again? Right.
I already said /multiple times/ that the other amazon services are well worth it. I only said EC2 /in my mind/ is not worth it, unless its for your overflow. If you have a static need for a server- As in, a webserver that will always be on, Then you are better off getting a dedicated. The bandwidth will cost less. The server itself will cost less. You will get more processing power.. You will get more RAM.
You will not get instant provisioning. But then again, I dont randomly change the specs on my webservers now either way.
340 dollars. If 340 dollars matter in your equation then I think we're talking about different things here.
I only said EC2 /in my mind/ is not worth it
Actually you come across like talking ultimo, just like in your next sentence...
If you have a static need for a server- As in, a webserver that will always be on, Then you are better off getting a dedicated.
See? That sentence simply doesn't work in that general form. If you have a need for a single webserver then yes, EC2 is probably not cost effective (although it can be even there, depending on the use-case). Once you're talking about a deployment with multiple servers the picture changes. Not always in the favor of Amazon but often enough to refute such a general statement.
Looking at it again it seems like all your arguments come from the angle of renting a single server. - In that light you make some sense. But please also see that amazon starts to make a lot of sense in the mid-range and up, in many situations.
I did a rundown a few posts up where I price out the need for a fairly common php/mysql website that would cost 600$ a month on ec2 to run, yet it only costs 200$ on dedicated.
Also, Im not talking about single server setups only. Show me a scenario where its cheaper to run your entire business on EC2 than a mix of Dedicated + EC2.
Seriously, I want to know of one situation where it would cost LESS money to run my entire 24/7 business on amazon then on dedicated or a mix of dedicated and EC2.
Thanks for pointing that out, my bad. Edited it as to not collect karma for false figures. I stand by my point, be it $140 more or less.
Also, Im not talking about single server setups only. Show me a scenario where its cheaper to run your entire business on EC2 than a mix of Dedicated + EC2.
Please see my first comment in this discussion where I said exactly that. It's exactly this mix which makes the most sense in most cases.
Also, Im not talking about single server setups only. Show me a scenario where its cheaper to run your entire business on EC2 than a mix of Dedicated + EC2.
I never claimed that - but yes, even those scenarios exist.
If your business is number crunching then EC2 might be a good deal. Likewise if your business is strongly saisonal (e.g. ticket sales for sports events or such).
Moreover there are many scenarios where EC2 enables things you could not realistically afford otherwise; do you think dropbox could have realized their storage and scalability needs on conventional hardware?
While traffic might be "ridiculously" expensive on AWS, storage happens to be riciculously cheap. You don't get $.10/GB "infinitely" scalable random access storage elsewhere.
And finally, closer to your usual PHP/MySQL shop, in many companies the savings due to flexibility and availability simply outweigh the markup on the hosting bill.
You keep throwing figures in the 3 digit range. Sorry man, but your admins burn that per hour while they reinvent all the little infrastructure things that amazon already has.
Trust me, not all of amazon's customers running those 1mio-or-something instances are idiots.
Im talking 3 figures because thats what I assume most startups need for their twitter/facebook/whatever app. It only becomes even more ridiculously expensive to stay with EC2 once you get bigger. You cant just say "well, once you are big its better!" Its not. Once you get bigger, the downfalls of relying on ec2 (for anything besides overflow) are even worse. The more servers you have and the more bandwidth you do, the more cash you throw out the window.
In fact as you get bigger you would be irresponsible to your shareholders to rely on amazon and not hire your own engineer. You know why? Its cheaper to have your own engineer and manage xx servers on colo then it is to have amazon. Not to mention you will still need a server guy once you get to that size anyways. Theres a reason you dont find any sites in the top 100 running their businesses entirely off of amazon. Its not financially prudent. Its a easy way out and not the /right/ way out.
Also, we are not talking about s3, simple DB or CDS, so stop bringing that up. We are talking about EC2. You dont get to say that EC2 is awesome because S3 exists. They are separate products.
I said multiple times that 1-3 server shops are irrelevant for this discussion. At that scale you can use anything and it simply doesn't matter either way.
Since you seem to have mild reading comprehension issues I will spell that out for you: Neither will a profitable business break a sweat over paying 500 more or less on hosting. Nor do many of the benefits of EC2 come into play at such a small scale.
So either throw out some stats and numbers or stop bullshitting people. There are enough people blindly drinking the cloud hosting koolaid without the fake facts.
I don't like your tone. I explained four possible use-cases in my last comment, I don't think I used "fake facts" anywhere. If you are immune to basic math then that's your problem, no need to get personal.
Its cheaper to have your own engineer and manage xx servers on colo then it is to have amazon.
I conclude that you simply have no idea what you're talking about, much less about the figures involved. Furthermore an EC2 instance apparently killed your pet dog at some point in the past, which now prevents you from any rational thought in that direction.
Upgrading hardware involves more downtime, and it's not anywhere near as easy to scale back down if I need to.
It only starts sucking when you actually have to serve files and do it quickly.
Shop around.