Comment by jackowayed | Hacker News Reader

Comment by jackowayed | Hacker News Reader

Parent Full thread

Heroku uses the same model. They're doing quite well.

GAE has problems, but I think the root is just how unique everything is. That manifests itself in people using a datastore that they don't understand, with Google expecting them to know how many writes an action will take and whether that feels like the right number of writes or two orders of magnitude more than if they made a different decision about how to store their data and solve their problems.

It also manifests itself in the lockin that Heroku mostly avoids (which is a huge problem if some subset of users get to a point where they realize "whoops, this would be much easier if I could do things Google won't let me do, time to leave").

I think a good counterexample is Engine Yard and GitHub. Engine Yard had a somewhat limited offering (especially for what GitHub was willing to pay) that didn't really fit with GitHub's heavy direct disk I/O. (Most Rails apps almost entirely read and write from the db, but GitHub does a lot of direct operations on the git repositories.) But GitHub was still just a Rails app, not an app for some specially-designed Engine Yard framework. So it was fairly painless for them to decide to solve the problem in a way that didn't fit with what Engine Yard would offer them and migrate to their own hardware. It wasn't easy, especially since they weren't solving an easy problem, but at least they didn't have to replace their database.

The GitHub example is the reason why I never wrote any code against GAE. The lock-in into their data store is a hard pill to swallow and was an early warning sign.

I am not familiar with the internals of Heroku and don't know how they solve the problems I outlined. Maybe someone else can elaborate.

Heroku runs a process for each application (or multiple). The first one for an application is free. But just like with GAE, they'll spin it down if it's idle long enough. So I know that if I go to a site of mine that no one has visited in months, where I'm not paying for extra instances, it'll take a few seconds to load. But having to run those free instances doesn't kill them because all but the first one for an application is paid for at 5 cents/hour, which presumable is enough margin to cover the free instances, and then some.

But they run normal applications. It started out being any Rack (Ruby web standard--Rails, Sinatra, etc) app; they've expanded into other languages now, but it's always some open framework that they're running for you, not something they own and keep proprietary. They give you a normal Postgres database. There are a few restrictions that you might not have on your own hosting (like a read-only filesystem). But you could basically take an app running on Heroku, install a webserver and Rails, install Postgres, make sure any config you had was the same, and run it.

Another huge advantage Heroku has is the ability to give other people access to their datacenters, since they're just running on EC2. So there's lots of Addon services that can add various pieces of functionality, like hosting a different database, many of which are only tractable because they're also hosted on EC2 and thus have very good latency to Heroku servers.

This means that you're not stuck with Postgres. If you think a part of your data, or all of your data, would be better stored in Mongo or Couch or Redis or flat files on S3, there are hosted services for that, and you can even deploy your own solution on EC2 if you'd rather. This leads to nice halfway solutions where you use Heroku to have super-scalable application servers, and maybe to manage your main relational db, but then you can tack on other things where that doesn't fit with your problem. Now, if you're running some of your own EC2 instances, you're losing some of the "never have to worry about hosting again" value of Heroku, but at least it's possible. It could be a temporary solution that keeps you above water while you migrate off of Heroku, or maybe you decide it really is the best long-term solution.

Thanks for the great overview.

It's interesting, since GAE charges $0.08/hour per front end instance, with 28 front end hour instances free per day. However, I once helped someone debug an application that on average gets one hit a minute, and yet manages to max the 28 free hours and then some. Closer examination showed that GAE was having these instances hang around for much longer than seemed necessary after they fulfilled the initial request.

So either Heroku has more magical magic than GAE, or there is some other kind of efficiency that they are tapping into. One thing I can think of is that possibly Heroku is more conservative with spinning up extra processes, preferring longer response times.

I may be misunderstanding what you mean but heroku does not spin up or down any processes, the amount of workers you have is controlled entirely by the end user and you will be billed by the worker regardless of how much traffic you're getting. I'd venture that 95% of heroku apps have a single process, and that 95% of the time those workers are in some sort of suspended state.

If you have 1 dyno, heroku spins that dyno down after 20-30 minutes of inactivity.

Also, there are services out there that will monitor your queue depth and increase your dyno count for you. Or, you could use the heroku gem to do that yourself.

Depending on how long it takes a call to be dealt with it might be worth telling GAE to never spin up more than one instance. Sure, the occasional user will have to wait a couple of extra seconds to be dealt with, but you won't have to worry about cost.

Reply on news.ycombinator.com