How to Improve Rails Application Performance on Heroku | Hacker News Reader

How to Improve Rails Application Performance on Heroku

blog.bignerdranch.com

46 pointsvanstee25 comments

I used to love Heroku but it's just too darn rigid. I hate the fact that everything is done behind the velvet curtain and I'm just assured it's "going smoothly".

Last week I finally bit the bullet and bought a VPS from DigitalOcean and installed the bits and kibbles needed for Rails apps. Specifically Nginx and Passenger. Was it easy? No - but now I have total control over my machine, and my app (which let's face it, will probably never need to scale for millions of users at a time) is in my complete control.

Heroku doesn't even have a local filesystem, what's up with that?

There really is a missing link between heroku and aws where one would have more control than on heroku but less devops problems than on aws. And if done right, people would be throwing money at it. I guess that is that elasticbeanstalk and opsworks are trying to solve.

This is what I thought Action.io would be, but after a support ticket where I asked them about this use case, they said:

===

"Hi Sergio - Thanks for the note and glad you like action.io.

Please keep in mind however that Action.IO is not a hosting platform - we are a cloud development platform that works with hosting providers such as Heroku, AppFog, EngineYard or Google App Engine.

You can read more about how you can deploy to Heroku from Action.IO here: http://help.action.io/customer/portal/articles/1011326-deplo...

===

So like you said, someone fills in this hole, and a lot of people would love pay for it. I don't need to add even MORE services for something as simple as file storage for uploads.

I had heard of Elastic Beanstalk but not OpsWorks. Thank you for mentioning that.

I haven't tried it, but Cloud66 is another provider in this middle ground (they don't host, but provide you the devops tools to do so on amazon, digital ocean, etc.).

I've heard of Cloud66. Is it any good?

I haven't tried it. It does look like a great service. One thing, however, that I don't think it provides is auto-scaling (i.e. provisioning additional servers or removing servers when they are (not) needed). If they were able to provide that, I think it would rule out a need for Heroku completely.

Is passenger the best way to deploy a rails(postgres) app on an EC2 instance? I have been trying to find a good solution for this for a cpl weeks now.

I've tried just about every server solution for rails and in the end settled on passenger + nginx. It's easy to setup/manage. And more importantly easy to automate setup/management(I use puppet for all my servers). It's also simple to monitor. There are almost certainly other stacks that are faster but the difference is small enough that I'm better optimizing elsewhere and saving the headache.

I know I can google and figure this out, but how exactly can I deploy a rails app with git to an EC2 instance. I want to do something like I can do on Heroku(git push heroku master) with an EC2 instance.

You can use Amazon's elastic beanstalk http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create...

I would like something similar that can be used for any VPS. So if I get a box from Linode or Rackspace, for example (or even a real server), is there an easy way to have ngnix+passenger+git+backups etc setup and configured in a secure way without having to manually do it all myself?

Depends what you mean by "best". It's certainly the simplest, and performant enough for most cases.

By best I mean simple and powerful enough that I dont need to worry about till at least a hundred thousand requests a day.

What was it that caused you to change?

"I used to love Heroku but it's just too darn rigid. I hate the fact that everything is done behind the velvet curtain and I'm just assured it's "going smoothly"."

I've admired their marketing and infrastructure, but have never been a fan of heroku specifically, precisely because of the "everything behind a curtain". But I felt that on day one - I never "used to love" Heroku.

Essentially the number one reason I left was no file storage. I could deal with things being hidden from me, I could with paying a premium price, but I could not deal with using yet another service for something as simple as file storage.

I wish Heroku would add static file storage to their roster.

You may have complete control, but you also have a lot more responsibility; for tuning your software, and keeping it up to date. I recently got bitten by a bug with an out of date version of apache, where the whole server would become unresponsive under heavy load; trying to update it without bringing the entire server down was a fairly stressful operation.

There is a local filesystem on a dyno, but it's ephemeral. They are ephemeral for a good reason too: http://12factor.net/processes

I know about that. Still not useful for 99% of the use cases where you would need ot save files.

Doesn't this overlook the issue of Heroku's random load balancing system?

This article seems to suggest that the number of Dyno's required for more load scales linearly with load but, as Heroku now admits, the routing is random and therefore scales much less efficiently.

I found this great simulator that someone put together to demonstrate the issue http://ukautz.github.com/pages/routing-simulator.html

If I'm using it correctly, in his first example using 82 dynos would actually result in requests taking over 9000ms. It takes about 150 nodes to get the average wait times down to around 600ms.

Parameters used:

  1. Nodes 82 
  2. Process/Node 1
  3. Requests/second: 338 (20300/60)
  4. Duration request min/max 243
  5. Timeout 100000
  6. Time multiplier 2.00 (not exactly sure what this does)

Using multiple threads per dyno does help considerably. I guess it's equivalent to multiplying your whole dyno pool as well as having intelligent routing within the dyno itself, but problems still crop up if you have slightly inconsistent request times.

Also, one problem that seems to be overlooked is the possibility that multithreading within the dyno itself is less efficient than threading across many dynos since the threads may have to compete for resources. I can imagine that performance may drop across Heroku's platform as a whole as customers switch to threaded application servers and reduce their Dyno counts.

The comment has a better point, use Blitz. I think that service is invaluable. It will push your service to the max and to a point where most applications, if seeing real traffic of that level, should be making hands over fists.

I use it to crash my own hobby playground regularly. A simple Flask app (Python) I wrote can handle ~120 req/sec before it biffs it hard (all linked back to db connections allowed on the free Heroku plan.)

As someone pointed out in the comments section, Puma (http://puma.io/) is another alternative.

tl;dr: Use New Relic and use Unicorn. Weren't these two things that the RapGenius post specifically mentioned as problematic? (Cost and memory constraints, respectively, IIRC.)

Heroku offers the Standard New Relic tier for free, so there's no reason not to use it. The pro tier has definite advantages, but isn't required to do what is being described in the article.

As for unicorn, unless your application is a real memory hog it should be possible to run 2 workers, which is a big improvement over using thin. But this is now the standard advice being given by Heroku, so there's not much new here.

Does anyone know how copy-on-write forking plays into the 512MB limit on Heroku?

If your app takes up 256MB, does that mean you can only fit 2 workers? Shouldn't there be some savings due to copy-on-write?

Isn't it better to use percentiles for metrics like response time? Response time might not fit a normal distribution.

Reply on news.ycombinator.com