Hacker News Reader

Comment by RyanZAG | Hacker News Reader

Not sure how comparing different versions of rails shows if its 'fast' yet. I'd think setting up some request target and seeing how far RoR is from that target with each version would be better. Or comparing RoR to other platforms is also a good bet, eg http://www.techempower.com/benchmarks/

I just don't really see how this answers the question other than 'ehh I think it looks a bit faster than it was awhile ago?'

EDIT: Also interestingly, JRuby seems to be slightly better on the small instance EC2 test at techempower than regular Ruby (small instance should be similar to a laptop?). Yet in these graphs it's much slower. Anybody know the reason?

View on HN

Amazon says their "m1.small" instance has 1.7 GiB of memory. That's a lot less than most current laptops.

Maybe JRuby uses more memory in total, but can handle more concurrent requests per MB of memory used? That could explain the discrepancy if it's true, though it's just speculation on my part.

Another theory is that the benchmarks are too application-specific. Rails is a complex framework with which you can build many different kinds of applications, each with its own performance characteristics. Conceivably, application A could perform better on JRuby, while application B could perform better on MRI. Depending on which application you use for your benchmark, either interpreter could appear faster.

This calls into question the utility of benchmarking an entire framework. It might be better to do micro-benchmarks, with each one testing a specific scenario. Scenarios could include, for example: A request that loads 1,000 database records into memory; a request for a static image; a request that queries a many-to-many relationship; a request that saves an uploaded file to disk.

The thing is, those are all I/O bound operations and in the case of the databases most of the code you're benchmarking is native extensions, so I would expect the difference to be negligible.

Testing whole Rails apps really does seem like the best way to get good data. The interesting speedups are likely going to be in places like string matching / string copying and other places that don't involve network access.

>EDIT: Also interestingly, JRuby seems to be slightly better on the small instance EC2 test at techempower than regular Ruby (small instance should be similar to a laptop?). Yet in these graphs it's much slower. Anybody know the reason?

Thanks to some community experts, we're now using TorqBox (TorqueBox 3, built on Undertow [1]) for our JRuby tests. The JRuby community seems satisfied [2]. :)

[1] http://torquebox.org/news/2013/12/04/torquebox-next-generati...

[2] https://twitter.com/headius/status/413021218728443904

Ah, that explains it. Pretty massive difference really. I guess that makes Torquebox+JRuby the best way to run ruby code? I wonder why the RoR community generally ignores it? Maybe it's just a timing thing and as more companies learn about Torquebox they will swap?

I'd wager that it's the TB/Undertow code running most of the time on that benchmark. On real-world benchmarks the difference is going to be a lot smaller. Not saying that you shouldn't run your JRuby apps on Torquebox - it's a great choice.

Indeed. In our tests, the plain Rack implementation is extremely impressive, but that is probably because there is so little Ruby code being run [1]. By contrast, the Rails implementation on JRuby is just a bit better than plain Ruby, but that is an improvement versus our previous rounds that were not using TorqBox. Previously, we had been using JRuby deployed on Resin, and the results put that JRuby+Rails combination slightly slower than plain Ruby+Rails. For what it's worth, Resin is fast as far as Java Servlet containers go.

Since TorqBox/TorqueBox 3 is very new, I don't suspect it's being used much yet.

[1] https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...

Reply on news.ycombinator.com