I'm not familiar with Rails. Could you explain what you mean in layman terms? Why is code being reloaded on every request by default?
As of Rails 3.2 development mode watches for file changes and attempts to only reload those files, but it's still a significant performance issue.
By default all Rails applications start in development mode, so one gotcha of benchmarking Rails is that some people will forget to set the mode correctly. That said, from the setup code[1] (line 14) it looks like they were running passenger in production mode. The max pool size seems excessive, especially when running on large ec2 instances, but I'm not fully convinced that it's out of line.
[1] https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...