Comment by LogicX | Hacker News Reader

Comment by LogicX | Hacker News Reader

This is an amazing article. I intended to write such a blog article myself.

One aspect I intended to cover (and will now do so here) is that of cost. I get very frustrated by cash-strapped startups which from day one are expecting to be 'web-scale' and need to turn up machines at the drop of a hat. Lets get real. I HOPE you have to worry about that... much, much later.

I'm quite pleased you started your article mentioning solutions like heroku. Unless you have some special-needs not met by a PaaS, this is where you should start. You should be writing code, not managing servers (this coming from an operations guy who has been managing servers for 12 years, and worked at a webhost). Once you scale far enough that its worth hiring someone to deal with the knowledge required to deal with the maintenance of managing your application stack, OS updates, security, etc -- THEN move on. Not a moment before.

Cloud servers are realistically not the price/performance/low-maintenance solution for MOST startups. You should get a VPS (Linode) or dedicated server (from a reputable company which can offer quick SLAs on replacing parts - like 2 hours at voxel.net). Dedicated servers are cheaper than you think. I pay voxel $180/mo for an 8GB quad core 1TB box on 100mbit/sec. It outperforms servers costing twice as much in EC2 - and thats not counting bandwidth or storage. Concerned about reliability? Buy TWO - In different datacenters. -- You're STILL saving money, and you have the exact same level of maintenance overhead as AWS (OS, Updates, full application stack); while reaping the performance benefits of bare-metal.

You do NOT want the headaches of colocation. You cannot pay your staff enough to stay local to the server 24/7 and the cost of extra parts on hand to make up the money over dedicated.

Your startup is not Google, so I won't get in to having your own datacenter. (Well done pointing out that they're not getting advice from your blog)

I had some poor experiences with dedicated servers at well known providers. Essentially if the hardware fails, they'd rather fix it than get you into something new so you can be back up and running. My site ended up offline for several hours, several times, so that some tech could run hardware diagnostics, etc.

EC2 may be expensive, but if anything goes wrong, I can boot another instance in seconds and abandon the old one. They fix it on their own time.

Sure, I could buy more servers so that one down one doesn't affect me, but that's twice as expensive. On EC2 I don't have any penalty for abandoning an instance for any reason. I've actually moved data centers in EC2 when the performance profiles were better on the other side for the same size instances. It was a temporary difference, but migration was pretty much painless.

You're voxel suggestion may have a 2 hour SLA for replacing parts, but what happens when they don't know what is wrong?

That said, I agree that a person should try for the PaaS and evaluate all the options fairly. I tried Linode, several dedicated hosts, and then finally moved to EC2 on a previous project.

This is certainly true for the common case, but there are certain types of workloads where dedicated is the only way to go. Where I work we were doing a bunch of compute heavy work and we started out on EC2 and recently switched to dedicated hardware, which resulted in a 30-50x performance/price increase.

You can find the full voxel SLA here: http://voxel.net/sla

The nice thing about going with a larger dedicated host like Voxel is that they have extra hardware & servers on hand.

In fact, they even have instant-provisioning of dedicated hardware: http://voxel.net/voxservers

I worked for, and have used dozens of dedicated hosts over the years (I spend far too much time on http://www.webhostingtalk.com ), and can totally agree that these are common issues with dedicated servers.

To each their own; just sharing my $.02 perspective on the situation ;)

What dedicated hosts did you use?

We've been using Softlayer for years with ~100 servers and have never had this problem. No host is perfect but overall it's been a good experience and a lot cheaper than EC2.

I love Softlayer. Excellent facilities, hardware, network and people. I just wish their RAM pricing wasn't so insane... $200/mo forever to add a $80 8GB stick.

I agree, I love EC2 personally, their mini server plan(or whatever its called) is great for keeping a production version of a project running while its still in an initial phase (very low user base) and then you can scale easily without having to learn anything new as you can run basically things the same as you do locally (I run Ubuntu locally and mainly develop for Tomcat so it makes it really easy to keep things the same on there). I looked at Heroku but as soon as I realized it would be rather expensive just to get regular datbase access (so I could run straight SQL if I needed to, etc) it was out.

FWIW - you can run command-line SQL for all heroku instances: https://github.com/ddollar/heroku-sql-console

The micro instance seems incredibly slow for me running Rails. It runs out of memory while doing a git index-pack.

The micro instance seems to be routinely throttled, for random reasons. My vm was choked for days despite only pulling in ~200 visitors per day. Upgrading to the Small instance has removed all throttling.

I'm running a standard LAMP stack on it, nothing crazy.

hmmm, yeah I have noticed weird behavior a few times, however I run a Java Spring app on there and most of the time it works, its pretty much me that is using it though, I would think that Ruby does use more memory than Java though.

The problem I have with Ec2 is their configuration. They have small instances and then large instances. Nothing in the medium range with 4 cores, 2 GB ram, 64 bit etc. So if I were a small company and I need 2 medium instances (which IMO is what most people want) I’ll end up paying .34 * 24 * 30 = $244.8 in Amazon where as in Linode I pay $60 per month.

I run my staging boxes on Ec2 and production on Linode. The small micro instances are good for staging/dev with just $15/month. No other provider I know gives you a VM at such low costs.

Dedicated servers are cheaper for the same performance, so you can afford 2.

I agree that you should be writing code and not managing servers, but in my limited experience PaaS only replaces managing servers with learning the platform's APIs, learning idioms specific to that platform and working your way out of the lack of libraries you are accustomed to.

A lot of programmer overhead.

This has been getting better. The platform-specific knowledge necessary to get a Heroku app running on Cedar is pretty much limited to knowing you need to put "heroku run" in front of your rake tasks or command line utilities, and perhaps 3-4 sets of arguments to back up a database, restore one, and restart your app.

Compared to the amount of knowledge necessary to effectively and reliably run a dedicated server, it's honestly pretty trivial.

I've had to rewrite gems and Rails code to work properly with Heroku. The biggest drawback to Heroku IMHO is the read-only filesystem. It makes simple tasks like creating ZIP files on the server more challenging.

They've actually moved from a read-only to an ephemeral file system with Cedar. You can write to your hearts content, but can't really rely on the files being there. In most cases where you'd want to write files, you really should be using something like S3 anyway.

Heroku's Cedar stack does not have a read-only filesystem. However, it is ephemeral, meaning it is not durable and it's not a guarantee that things you put on the filesystem will stick around between requests.

This happened to me as well. If I had to do it all over again, I would have started pushing to Heroku from the start.

I think you are implying that cloud servers are different than Virtual Private Servers. I am new to this, so forgive my ignorance, but how are they different? Is it just that effectively that with standard cloud server options, you can spin up new/multiple instances, whereas with a VPS, you've just got the one?

Or is it more about how it is all handled in the background?

So Amazon uses some of the same technology (Xen); but their packaging, support infrastructure, and pricing model make it very different than a VPS offering.

VPS you typically get bandwidth, storage, etc. included - and its 'uncomplicated' - You pay one monthly fee, it covers it all.

AWS can be viewed as both fault tolerant, and additional probability of fault due to the extra complexity they've built in (they've gone down for substantial periods due to hiccups in this additional complexity).

A good VPS provider will make the particular system you're on fault-tolerant on its own: dual power supplies, RAID arrays, etc.

AWS doesn't care about making any particular machine fault-tolerant, because their model is that you should spin up a new instance, and throw away the one that failed.

Thats great for a 'web-scale' enterprise that has invested the resources making their site operational for that mindset. For early-stage startups where dealing with a failed instance means their site is down until they create or restart a new one; the advantage goes to VPS IMHO.

One of the differences is how VPS and "cloud" is metered: typically monthly (may be prorated daily) for VPSs and hourly for cloud servers like EC2. You can spin multiple instances in both cases, the more differences come in play if you start using cloud-specific features like load balancer, cloud storage (ie S3), etc

Using Heroku is not free as a programming overhead as you have to bake in Amazon S3 as a dependancy.

That implies your users are uploading files. Certainly not the case for all startups, but yes, this could certainly be an additional overhead. Still, compared to running and configuring a VPS, thats nothing.

Also, if you're handling user uploaded content you shouldn't just be writing it to local disk. You're going to have to figure out how to do off-instance blob storage at some point if you scale beyond a single server or want any redundancy. S3 is just about as good as it gets for blob storage today.

Reply on news.ycombinator.com