Why Wasn't Ruby 3 Faster?
fastruby.io
fastruby.io
When I was a junior engineer, one of our demos to a client failed because we had a group of 20 people try to use a server that had a highish latency connection to a database, and using synchronous IO meant that a 50ms db round trip for each executed query, with each request running maybe 5 or 10 queries, on a VM with 2 cores and one Rails process per core, would queue up incoming requests until they hit the 20 second mark and started timing out.
Bad concurrency because of evented IO is not exactly the same thing as being slow, but it manifests as much the same developer experience, so it gets misinterpreted and Ruby gets a reputation for being slow.
The team could have booted up more processes. Isn't that the answer?
Or have I misinterpreted your comment?
Everyone dropped the concurrency ball hoping that was someone else's problem. Its the dev team's fault because everything is ultimately the dev team's fault. But it would also be great if it was fixed in ruby + ruby's ecosystem.
Cause running more than 2 processes on a 2-core server just sounds weird.
I thought Node's doctrine was "One process per core", and (async) Rust and Go seem to do fine with managing their own thread pools. I haven't _tried_ to do a realistic benchmark of Hyper, but I've never seen it struggle with concurrency either, even on long-polling applications where concurrency was the norm for all clients.
More processes would work, but it doesn't seem like the right layer of abstraction for 2015, now that Nginx and Node have made the event model so popular.
The example given was a Rails app. I don't think one process per core is common for Rails deployments but I'm not sure.
- overprovision resources for big demos by a significant amount
- test your demo before you give it
- deploying pets (servers you ssh into and lovingly configure) instead of cattle (instance abc123 that's created automatically) means you can't scale them as easily
- if you have concerns about a big demo, particularly ones that could be fixed by spending $10 on EC2 instances, you should fix them yourself instead of emailing the tech lead and assuming he'll handle it
- spinning up processes with phusion passenger is awful[0] and we should have been using something threaded like puma
That being said, the same footguns apply at various points as you scale, because Ruby's GVL still exists, and because even though threads are lighter weight than processes, they're still relatively expensive to switch between and to maintain in memory compared to e.g. Go/Elixir lightweight threads or evented IO.
Unfortunately (or perhaps fortunately) the problems with threaded IO start applying well above 20 concurrent users, so I don't have any fun stories about failed demos involving Puma instead of Passenger.
[0] Specifically, Rails processes take up a lot of memory, none of it is shared, context switching between processes is pretty expensive, and so you're limited to a smaller number of processes that can become IO-bound very easily
Another optimization is using jemalloc. Going to try this soon with Sidekiq workers.
This is a really common misconception. Both switching between threads and processes is _extremely_ fast compared to what web applications do.
An exceptionally fast web response is generally <1ms. Linux thread context switching takes ~2 microseconds. Forking a new processes takes ~20 microseconds.
It's also not a fundamental restriction on threads vs Goroutines either. switchto is very nearly as fast a Goroutine context switching https://www.phoronix.com/scan.php?page=news_item&px=Google-U...
Not true. Context switching pthreads is orders of magnitude faster than any Go/Elixir concurrency mechanism.
So-called "lightweight" threads exist to fix the concurrency problem with garbage collection in scripting languages. Without garbage collection pthreads are fast enough for any problem.
That's the original advantage over OS threads.
Not 200,000. A single process wasn't able to handle 20.
You'll be hard pressed intentionally writing code that performs that slowly in a framework in another language.
Ruby by default has an incredibly long list of "you can but you shouldn't" and things you need to be aware of, especially around concurrency.
Other languages are mostly fast by default. When you follow some nodejs/express/whatever introduction tutorial and implement your first application in the most naive way possible, you'll end up with something that will easily scale to hundreds if not thousands of users out of the box.
Do the same with ruby (or even with some python frameworks, to be fair), and stuff can break at less than a hundred users.
And that's why yes, you can blame that on ruby. Or at least ruby how it is commonly taught.
That aside, a ruby script that can't handle 20 req/s on 2 vcpus seems rather suspect. I'd be really curious what that code looked like. I'm doubtful that it has much to do with the language and not the code design.
I have never ever been concerned about concurrency in Ruby, or Python or PHP. Just deployed the number of workers which would be able to serve our peaks.
I do write high concurrency application though, but those are not web apps and are not written in these languages.
The example said there was 0.5 seconds of IO per request so each process could serve 2 requests per second. 2 processes would serve 4 requests per second. The mistake was running only one process per core. 10 processes would server 40 requests per second, and the cores would still be mostly idle. With rails you always run as many processes as memory allows, not one per core.
Now that rails is thread safe you can have each process run a few threads. If you ran 10 processes and 4 threads per process you are up to 120 requests per second.
If you really need massive concurrency that isn't anything special. But that is more than enough for 90% of the web apps out there.
Serving only 20 users only has to do with how many workers were assigned.
I actually implemented a high traffic RoR payment provider on minimal infra ( 1 small VPS ), and benchmarked a couple of Python CMSes and compared them to WP.
I'd like to see your results if you still have them.
Anyway, I like Ruby more than any of those languages, so hopefully speed in 3.0 is at least on par with Python and Perl now.
“Perl 6” hasn’t been a thing for more than a year.
As for Ruby 3, the concurrency improvements are here today. The Actor-model parallelism of Ractors is here, if experimental, today. JIT improvements are here today.
Yes, there's also more stuff on the horizon building on that, but it is simply not the case that efficiencies from Ruby 3 are so e kind of continuously receding mirage.
Processors have only been going sideways for a while and python makes this a pain.
Elixir and Rust are a little less painful in that regard.
Yes, but this article is about speed, not concurrency, because speed is a thing 3.0 didn't do a major improvement to from the last 2.x release.
There's no need for a “Why didn't Ruby 3 improve concurrency/parallelism” article, because it did.
Fiber#scheduler addresses your IO issue, and Ractors (while still experimental) are aimed at the parallelism issue.
I do not have experience with Ruby, but I think it's similar to Python (in performance) and even if it's let's say 5 times slower (which I don't think it is), a webservice on a simple machine (workstation) in Ruby can easily serve north to 1k (or more) requests per seconds (behind nginx or apache or whatever), and in no way would 20 users be a problem.
It doesn't matter if that's async or not (note, not being async does not rule out concurrency or parallelism), but there was something seriously wrong with your application (if you have to serve long-running tasks, you have to make a msg queue anyway and shouldn't do that in your HTTP response directly). You may gain from async if you are IO bound, but with 20 users you are not IO bound, believe me.
You're absolutely right when you say there was something seriously wrong with the setup! The problem is that it was more or less what you'd get if you set up a default Rails app behind Passenger, and Passenger did this because some of the major Ruby gems weren't threadsafe (e.g. Rails, until 2008), which comes back to concurrency problems with the ecosystem.
The 20/20 hindsight is not meant to retroactively solve their problem, it's mean as a counter-argument, that the problem was their design, not Ruby.
It sounds like the concurrency was set much too low. That's not limited to the configured max workers but there will be similar settings for the web server, database connection layer and the database itself.
A similar thing would have happened with C++ or Java if there was a connection pool of two and database queries that took 500 ms to complete. I've seen worse mistakes in producion code.
The lesson to learn from this is probably not that goroutines makes everything faster, but that you have to understand the tools you use. Also, test demos. And always keep a backup plan. That's what Steve Jobs did, relentlessly, for all his demos.
You can get around it by using async instead of blocking, using fibers which don't block, or using a thread pool so other threads can keep serving.
I'm not experienced with Ruby but it sounds like Ruby doesn't do any of those
Modern ruby can do any of those, though in 1.8 or earlier it could do fewer, and less well.
The typical approach for Ruby, Python, and similar languages is just to run more processes.
It's hard to remember what we had to do back then, but Passenger was probably there. My first though: did you use a connection pool and a C driver to make parallel requests to the database?
I really, really dislike Rails for any number of reasons, but not having spent the time to understand how to set it up is not Rails fault.
JS has an equally bad concurrency story as Ruby. Maybe even worse. There are no threads, you can only use a single core per instance. In practice the problem is ram. JS already uses a lot of ram compared to Java and running an instance for each core makes it way worse.IMO only crazy people run JS backends when there's so many other similar options
This is absolutely false. Yes, JS is single threaded, but all IO is async and happens via a worker pool. On top of that it is 3-10x faster than Ruby for most tasks due to the insane amount of development effort that went into the browser runtimes, it’s often within 50% of C speeds.
And then the entire ecosystem is async. This is why you’ll see JS frameworks near the top of TechEmpower benchmarks achieving massive concurrency numbers, and also why nobody recommends any kind of CPU-heavy workload in node.
Imagine Ruby came with Puma built-in, and all libraries only exported async interfaces.
JS is strange because of all the optimization. It's technically multi threaded with IO but in a way that you only control the "main thread". Reminds me of old UI work
Well, it also has Workers, which is.
Basically almost every high performing server atm is running on some form of event loop / cooperative multi-tasking. Ruby Puma, Gunicorn, rust’s tokio and async/await, goroutines... Java is probably the big exception here (if you’re not using VertX et al) and we all know the pain of thread safety.
Node has both blocking and async versions of a lot of IO operations. But async IO is more commonly used certainly. The real question might be whether a thread pool backed async IO implementation makes your program do concurrent IO better than a GIL + blocking IO calls that release the GIL, it's not obvious to me.
(Doesn't Node have threads too nowadays?)
Node’s threads are for cases you really want to do heavy computation without resorting to another language. All the I/O is still backed by LibUV and an event loop, threads don’t really offer any improvement there.
As far as threading, Java is adding userspace M:N threads which will be far superior in most use cases to C#'s async. And unlike async it gives an automatic performance boost to most existing code
C# has a long history of breaking changes. One of the reasons Java is more popular is 90% of decades old code runs fine on the newest version. Java's backwards compatibility is absolutely unmatched.
I'm fairly sure GIL is released when waiting for external I/O, at least all the decent C database libraries do so. So in your case threads would have help quite a bit.
In general yes, the GIL is imho a problem and I do not like it, but at the same time there are ways to configure applications so that they do not mind that much.
For non-thread safe things (are there still any?) there is always the goold old way of just running multiple processes (with preloading the memory usage of even 10, 15 puma workers is often acceptable).
Ruby traded safety (GIL) for absolutely speed, although it should be noted that Ruby's C interfaces do allow you to op out of the GIL, which means if speed is absolutely paramount you can write a Ruby C extension to opt out of it.
I've done it in the past, but OTOH if you're to that point maybe Ruby no longer makes sense.
It's almost always worth doing some concurrency/scaling sanity checks on anything you're preparing to demo IME. There's plenty of ways any particular app, no matter what language or framework, could unexpectedly fall down when facing 20x traffic. If not from the language itself, perhaps from various external dependencies. It may be true, particularly at some previous times, that doing the most naive thing in Rails hosting was significantly worse than doing the most naive thing in Java hosting, but still, always worth checking.
The 50ms DB round-trip 10x per request smells funny. Most queries to a closely located DB with demo app data should be more like 5ms or faster. Maybe there's good reason for it to take so long, I'm not sure. Perhaps the DB also had some slowdown from so many of those queries being executed concurrently, or maybe it would have if you had had a faster Rails setup or used Java instead. All the more reason to always test it and not just assume it'll be fine because Java should be fast.
Each individual web request is still gonna be rather slow if it's gotta do multiple 50ms DB round trips. I don't know Java that well, but C# and Ruby both have pretty decent tools for making that IO concurrent in the scope of an individual request. You have to deliberately use it though, since only you know which queries actually depend on data from other queries.
I've been working professionally with Rails for several years. If I was deploying such an app now, it'd be simple and fast to say, I suppose I'll serve it with Puma. 2 cores, so 2 processes. 5-10 threads per process is pretty normal, maybe make it 20 here since I expect a lot of time waiting on DB IO. Now we've got 40 workers and it ought to serve traffic from 20 users fine. Assuming some other part doesn't fall down first.
> "Ruby 3 will be three times faster compared to Ruby 2."
The context is that he was rereferring to release day Ruby 2.0 vs Ruby 3, not current Ruby 2.4 vs Ruby 3.
I think the lesson is that managing user's expectations is critical to major releases like this. Even if you are well intentioned, it's very easy for a promise to be misunderstood, and once it's out there, its out there. It's very difficult to put the genie back in the bottle without giving the impression the project is having issues.
This whole thing reminds me of the whole KDE 3 to KDE 4 transition. “We never said 4.0 would be usable, but also it’s a big major public release, please use it, but also don’t complain about it being unusable”.
It seems that some developer communities always want to have their cake (the flashy marketing / sloganeering) and eat it too (faster ship cycles and shipping beta quality software as production ready because that’s faster).
EDIT: to clarify in Ruby’s case I’m not saying 3.0 was beta quality. But TFA goes through all sorts of gymnastics to try and avoid admitting that they tacitly let people assume there’s be a revolutionary speed bump, and smells a lot like retreating while declaring victory.
If you have children, and they have children, I hope that those grandchildren of yours grow up hearing stories about what a travesty the roll out of KDE 4.0 PLASMA Desktop was.
Here’s my entry into the annals of KDE 4.0 Plasma Desktop Release circa 2008:
It basically worked for me. I was a sysadmin, so abused my authority at work to sudo apt install kde-desktop on my mandated Ubuntu work machine. My personal machine was a Gentoo thinkpad running fvwm2, but I did use the excellent Kate editor and AmaroK media player. I was a glutton for punishment at the time.
I think more than anything, seeing how salty people were about the transition taught me the psychology of the downstream user, and how a classic “politics of power” narrative can be applied to almost any situation for rhetorical effect, appropriately or not.
It’s kind of an impossible quagmire. The developers were mostly volunteers working on what they thought was interesting and worthwhile, most users were probably fine and silent, and a percentage of very loud users felt very personally wronged, as if something immoral had occurred. They’re not wrong to give feedback, but the situation is that they basically have to beg volunteer strangers. And, broadly speaking, most of the people complaining didn’t come from a background prone to begging or being told no.
Another thing I chuckle to reflect on, is that iOS has been shifting towards a “KDE 4 AIR” look and feel since 2015. The latest iOS innovation: KDE4 style “Widgets” ;)
KDE is mostly driven by volunteers. During KDE 4.0 time, KDE didn't have any releases for a long time. That meant some volunteers were losing motivation, things were getting too long, etc.
As an example, KDE's Games collection was in a fantastic shape and ready for a very long time. Should they all have waited until things were polished? That would've taken 1-2 more years. KDE Games developers were not gonna keep waiting for that long.
While releasing KDE 4.0 was a mistake, the biggest mistake was distro's rushing to ship it by default. KDE 4.0 should've been optional/sideloaded. That would'v been the right balance.
However, the hype around KDE 4.0 was so much that distro's rushed it and the rest is history.
People expect if it could be 3x faster in those benchmarks, surely it isn't too much to ask for 50% increase in my Rails Apps.
https://youtu.be/LE0g2TUsJ4U?t=3242
If you watch that for 5 minutes, Matz makes it clear that a) it's a goal, not a promise, and b) he's basing this on ruby 2.0
I don't know how you can set expectations any clearer than that (other than maybe not to announce any goals ever, or chase down the 3rd parties reporting on your goals with a torch and pitchfork, demanding that they fix their lazy reporting)
Meanwhile, Ruby performance is right in line with Python[1], and in general people are not ragging on Python for its slowness.
If the Ruby world wants to keep growing and thriving what I think matters more is the ecosystem. These days Ruby is very mature, battle-tested, and has a rich ecosystem for web application development.
But this meme that Ruby is “too slow” has taken some of the air out of the balloon, as well as just becoming “boring old tech,” and combined there is definitely less active / fast-moving development work in Ruby libraries compared to ten years ago.
IMO the biggest problem with Ruby has always been the double-edged sword of Rails. Rails is the only thing that really made Ruby go mainstream, and if the language is going to really bloom it needs to expand beyond that. Ruby is a great high-level general purpose language, and it could be a good fit in all kinds of applications, but until one of those takes off we’re just going to keep reliving this meme of Ruby being too slow which came about around the peak of Rails popularity (circa Ruby 1.8.7 to 1.9.3) when it had much more truth to it than it does now.
[1]: I develop a lot of small applications in both Ruby and Python, and for curiosity’s sake often convert one to the other language and benchmark. There’s no perfect way to compare languages, but that’s a reasonably good one.
It's a lesson on promises. They shouldn't have made the "3x faster" promise until 3 was at the RC stage. Making it as early as they did pretty much set them up for failure. Maybe they didn't predict that 3's development would take as long as it did, IDK.
Hype the features you have, not the ones you will have.
I know the Ruby community doesn't always live up the the "Ruby Is Nice" standard, but when it does, I just love the language that much more.
Like, yes - all those rules make for good advice for doing that, but they're rigid and aren't the right choice for all situations.
(https://rubyonrails.org/doctrine/#optimize-for-programmer-ha...)
I think the reality of ruby is that it's a language for people who care more about the feel of the language than its performance characteristics. This may sound like I'm ragging on it but I think there are many cases where optimizing for developer comfort is truly in the best interest of an early to mid stage startup.
But don't try to make it into something that it's not like you are competing with rust or something. Ruby simply is not a performance oriented language. That being said it's performant enough for a vast majority of all and medium scale use cases.
It hasn't been worked on for 3+ years now, but its an alternative to using MJIT.
You'll need to need to write most of the C function calls back into Ruby and have the JIT properly inline them to get equivalent performance.
You won't get an immediate jolt of performance but over the long tail, it will catch up.
What works really well for TruffleRuby etc might not work that well for a simpler optimizing compiler you could build and get merged into CRuby. LuaJIT and JSC are simpler than a lot of other "fast" JIT runtimes and that's not really the approach they take.
You're free to use the simple passes that the JIT there provides, but the more expensive optimizations are also available as the Java JIT has a tiered optimization model. [2]
All the optimizations found here [3] can be used along with the code-generator opts as well.
I guess the idea is to provide a full suite of pluggable components, pick and choose the ones that make sense for your purpose right now, and those needs can evolve over time.
[1] https://www.eclipse.org/omr/starter/whatisomr.html
[2] https://github.com/eclipse/omr/tree/master/compiler
[3] https://github.com/eclipse/omr/tree/master/compiler/optimize...
Something like Graal would never succeed with FOSS, late nighters and weekend programming.
Licenses are usually not where it hurts on corporation scale projects.
It is the same issue I've seen with jruby and rubinius back in the day: they claim it is as performant or faster than mruby, but then you port a simple rails app and it hardly even manages to start the server. Not sure what these ppl are doing for benchmarks, but it ain't representative
Let's be clearer: some people will have thought that.
Techie: Here is my business case for moving some stuff to Rust/Go
Approver: But one of the Devs said Ruby 3 will be 3 times faster.
Techie: It's more complicated than that, for these eight reasons.
Approver: But still. 3 times.
Often when people say Ruby Rails dont scale, they dont mean in literal sense it cant scale, it just means it is comparatively speaking expensive to scale. And depending on how you value multiple factors the term expensive is subject to debate and discussions.
Last I had heard, Rust was targeted at being a "Systems" language, ie. an alternative to ASM/C/C++, where Ruby is basically a webserver/backend language.
What makes you think Rust will be used as a "Ruby on Rails" replacement anytime soon?
I would probably reach for Java (or ideally Scala) for that sort of thing these days, but would love it if something like Rust could compete in that space. Maybe it can't, but I wish it could.
I think one place where Rust could be great for these sorts of things is low-horsepower devices like routers and managed switches where you want to have a rich web interface but can't afford the overhead of an interpreted language. Being able to ship a device with a slower CPU or less RAM/storage because its web UI requires fewer resources would be an advantage in a market that's already very price-competitive.
Just to disclose my biases: I'm of the opinion that you shouldn't ever be writing anything in a dynamically-typed language if you expect it to turn into a large program. Your initial development will indeed be rapid, but later on you'll be spending more time writing tests than improving your software... or you won't be writing tests and will be too afraid to make large changes without breaking something.
I can usually familiarise myself with an app in a matter of minutes and can get a working prototype up and running within an hour.
All this makes me really productive, and writing Ruby is a joy.
I think it is important to talk about this topic and performance. I agree that Ruby is plenty fast enough for many things.
javascript: function()%7B%20let%20i%2C%20elements%20%3D%20document.querySelectorAll
('body%20\*')%3B%20for%20(i%20%3D%200%3B%20i%20%3C%20elements.length%3B%20i%2B%2B)
%20%7B%20if(getComputedStyle(elements%5Bi%5D).position%20%3D%3D%3D%20'fixed'%20%7C%7C
%20getComputedStyle(elements%5Bi%5D).position%20%3D%3D%3D%20'sticky')%7B%20elements%5Bi
%5D.parentNode.removeChild(elements%5Bi%5D)%3B%20%7D%20%7D%20%7D)()
Thanks dang for all your work! cheers
edit: did i win the fastest comment to the bottom on hn? what do I get?
jokes aside, care to explain why folks?
In all seriousness, the results speak for themselves, when comparing to Ruby in terms of speed.
https://github.com/kostya/benchmarks
downvoted, apparently. As a Rails dev though, Crystal does look very promising, and as a Typescript dev, it has optional strong typing with a syntax that is much, much more appealing than RBS.
For the majority of us who use Ruby, we use Rails, and Rails is great. Crystal is irrelevant as a comparison to Ruby, because there is no Rails (I'm sure there's web frameworks, but they're not nearly as productive as Rails). Rails isn't perfect, and there's plenty of room for improvement, but switching languages and starting from scratch isn't the solution.
The biggest downside of Ruby compared to other languages (besides speed, but like I said, that doesn't matter here), is lack of static typing. That may or may not be a problem depending on your usecase.
I wouldn't say it's irrelevant, Crystal is already being used by ex-Rubyists and the ecosystem of crystal libraries are being built by them as well.
After 1.0 you would see more of this happening.
Lucky [0][1] by Thoughtbot is the closest to Rails in the Crystal world.
> The biggest downside of Ruby compared to other languages (besides speed, but like I said, that doesn't matter here), is lack of static typing. That may or may not be a problem depending on your usecase.
Then I shouldn't see any complaints about speed on Ruby if it doesn't matter, Crystal solves all of this built in, with the trade off of slower build times, but then again Rust also has slower build times as well.
But to each their own.
I'm not sure "Ruby but faster" is the point. I'd argue that it's more "cool new language that should be comfortable for anyone already familiar with Ruby".
Could be that. I've used Ruby for 10 years, I'm just used a dynamic language. I also dealt with a typed Javascript monorepo and it was not very fun to deal with when the types weren't working for whatever reason.
And your productivity on your team plummets as a result.
Rust also has slow build times [0], but it didn't stop people from using it.
[0] https://pingcap.com/blog/rust-compilation-model-calamity
If the rust syntax and borrow checker wasn't masochistic enough...
The solution is to start breaking up the monolith so you only compile sections at a time.
So when did you start using a Nokia 3310 as your dev machine?
It's good for people to know of alternatives.
So, No.
Personally, I'd sooner code in Go than in Crystal. The tooling is way better.
Mruby however would be my first choice for "Ruby but with compilation."
If that won't work, then I'm looking to Go next.
A pretty syntax on top of a clean object model is a key selling point of Ruby.
I agree with you about Crystal though. No interest in it.
Language designers all optimise for different needs and if they find an audience they are doing something right for someone even if it isn't you.
If I was to migrate away from Ruby, Crystal would not be a candidate.