Don't get me wrong, I think Rails is an amazing technology but it doesn't fit the use cases and demands our clients have today.
We run 100s of dynos for our service, but could probably run just a couple if we moved to elixir/otp.
On one project my team was convinced that Redis was our bottleneck. Profiling was showing that Redis calls were where the most time was spent. I spent some time looking at the redis-rb source and it does a lot of block nesting (4 or 5 levels for every call, IIRC). So, I tried replacing the redis-rb client with something a bit lower level and our throughput doubled. It turned out that all that block processing cost as much as the network call did.
I suppose this isn't all that new: many performance issues are really a "death by 1,000 cuts" situation. Oftentimes your profiler can lead you astray.
In situations where a blocking redis client like redis-rb doesn't interfere with other running code (synchronous web requests), I've found zero performance issues with redis-rb (latency caused by redis is measured in single digits, usually < 5ms).
My intention wasn't to rail against redis-rb. It's written very nice, idiomatic Ruby. The faster solution was decidedly less Ruby-like. I was just suggesting that even when your profile looks to be telling you the DB is to blame, you should probably prod a bit more.
ActiveRecord is as another case where I've hit performance issues. I had just swapped out most AR usage for Sequel, so my data on that is probably well out-of-date. But, for us, materializing rows as full DAOs was often a very costly process that could dominate the cost of the DB query.
I get that the Erlang VM is really nice for some things (I made my first contribution to it in 2004!), but I'm curious about the above statement, as I haven't had the time to dig into Elixir/Phoenix yet.
There are some things that you simply must use other tools for in Rails, because it can't maintain state like Erlang can. Once you start adding in these other things, you're no longer just a "Rails app" but you're really creating an entire system. Now you've got different things you potentially need to test, and you're adding in extra stuff to your system to monitor and measure the different parts of your system. You might do the same thing in Erlang/Elixir, but you also might not. You can potentially keep your entire "system" inside Erlang, either in one node or distributed. So right away simply having that option sounds like a huge improvement in maintainability. You've got everything running in Erlang, monitored and supervised in Erlang/Elixir, more easily testable using ExUnit or whatever.
And I think @bcardella talked about this in his RailsConf presentation (linked somewhere in the comments here).. but the performance you get from Elixir/Phoenix is potentially a huge improvement in maintenance. How much time have people spent in NewRelic or whatever measuring and tuning their Rails apps, getting the caching and everything working just right? People are writing Phoenix apps with zero caching that are performing better than Rails with caching. When people aren't having to go debug and diagnose these kinds of performance problems they get to work on actual features.
Again, some of this is just my intuition on it more than real experience to back it up. I've built a couple apps with Phoenix and I do enjoy it more. They're not big enough for the performance to really matter, but I do feel strongly that they're simpler because I'm keeping everything in Elixir code and not having to use, for example, Sidekiq in order to do anything async.
The one thing that comes to mind is that since each process has its own isolated memory space, Erlang's GC is not a "stop the universe" kind of thing, the GC can run in parallel with other processes.
(edit: again, this is just intuition on my part, I haven't remotely done the deep dive you're talking about)
It could be a case of not yet being optimized for the tests, but I was expecting much more impressive numbers out of the box (particularly after the full-court press on the boards and blogs).
Chris McCord talks about it here https://www.reddit.com/r/elixir/comments/48ke69/any_reason_w...
I used Erlang at the last place I worked and like it a lot, but Rails is pretty good too in my book.
- macros: fancy syntax and "magic" can be resolved at compile-time. Less work to do on runtime.
- templates: they get handled at compile-time, too, resulting in functions with blobs of binaries. This matters a lot, since a specific template-binary exists only once throughout the application and gets re-used whenever needed. This way you get near instant templating, instead of the usual string processing on every request.
- dispatching code like the routing that must be done on every request is like rails a DSL, but actually a macro, that gets compiled down to basic pattern matches, which is extremely fast.
- the concurrency model is extremely lightweight, handling more requests and/or doing more stuff per request is much more efficient hardware-wise. Indirect performance gain.
Just a few points that have a very noticeably impact, there are probably more. It boils down to the erlang vm's concurrency and pattern matching performance, plus elixir's compile-time macros and the fact that they treat strings as immutable binaries.
All well and good, but Erlang/beam code is not that fast.
> templates
Ruby's templates are parsed into code that stays in memory, too, so it's not like they're re-parsed each and every time.
> routing
Perhaps clever use of pattern matching helps here. But my point is: someone should dissect these things in a real-world-ish application to see what's actually true.
> concurrency
Yes, but let's be precise. Everyone knows Erlang's concurrency is way better than Ruby's. The claim was 'fast' though. As well as maintainable, which seems curious given that there are no really old Phoenix apps out there.
True, no doubt. You wanted a comparison why phoenix is faster than rails. And simply doing things at compile time reduces work at runtime. In Ruby, all Metaprogramming must be done at runtime, for example via method_missing trickery (been there, done that).
> Ruby's templates are parsed into code that stays in memory, too, so it's not like they're re-parsed each and every time.
This is actually not the same. A single immutable blob of binary (elixir strings) which is shared throughout the whole application can leverage hardware caching better. Jose could answer this probably better than I can do.
> someone should dissect these things in a real-world-ish application to see what's actually true.
Moz.com recently had fun with Elixir [1]. You may also have a look at https://www.youtube.com/watch?v=OxhTQdcieQE from latest RailsConf.
> The claim was 'fast' though
Well at least I know that BEAM processes are far lighter than eg: goroutines. Also there is a per-process GC, so no "stop the world", much smaller units for individual collects, and when a process finishes before the heap grows full, it can be discarded directly.
> maintainable, which seems curious given that there are no really old Phoenix apps out there
Fair enough.
Personally I'd not wait until a software stack is a decade old before even considering it. I have at least worked on Elixir/Phoenix projects for many months now on-off with multiple colleagues, and it was/is still pure joy. Aesthetically pleasing syntax helps (broken window syndrome I guess), plus functional programming style in general, plus phoenix' foundation in "plug" and the clear modularity. "Let it crash" with supervisors also is incredibly robust, and robustness in itself leads to less maintenance costs.
---
Stuff like "Erlang processes are lighter" matters in some contexts, but not in a straight up speed contest. It matters a lot when you start trying to handle a bunch of concurrent connections, so maybe that's where we're getting some of the claims from.
Functional programming is not an 'antidote' to having to go back and read old code or necessarily understanding it easily.
It references this:
http://sorentwo.com/2016/02/02/caching-what-is-it-good-for.h...
Which is interesting, but I don't see source code. 1.5-2X is pretty believable, but quite a bit less than "12 times" - which someone quoted here.
Depends on what you're doing though. Seems to excel when having low CPU intensive workloads but high traffic pressure from my experience.
Here's that slide, I believe: http://i.imgur.com/QzTCJS8.png
This too shall pass.
In a Rails system you've got Rails running, maybe across X number of instances with Unicorn or whatever, sitting behind nginx. If you want to have a long-running request or websocket you've typically used Go or Node or Elixir. If you need to store state between requests you may be using Redis. For background jobs and general async things you've probably got something like Sidekiq.
Erlang already has an http server that Elixir apps are using now, it's called Cowboy. It's not that Elixir is creating their own just because. One of the reasons nginx is often used for Rails apps is to handle static assets, because Rails just isn't as well-equipped to handle those. Cowboy seems to be fine with handling static assets, so a lot of people just take nginx out of the equation for an Elixir app.
Typically Rails apps have used something else like Node or Go or Elixir for long-running requests or websockets. Maybe that will change with ActionCable, maybe not. Either way, Elixir/Phoenix happen to be quite good at this already. Phoenix's equivalent to ActionCable is called Phoenix Channels and it's able to handle 2 million+ websocket connections on a single server, which is pretty cool I think.
Rails apps use redis to manage state between requests. Elixir apps just don't need something like this. Elixir doesn't have something different, it just happens to be easy to store state between requests already.
Rails apps may use Sidekiq or something to deal with background jobs or async operations. This is another case where Elixir doesn't have anything special to occupy that role in the stack, it just happens to be good at that already.
Most of these properties are things that Elixir inherited from Erlang, which has been around for years. I wish I had started learning Erlang a long time ago but unfortunately it just wasn't really on my radar. But really the only thing on this list which is a relatively recent thing is Phoenix Channels, and I don't think that's really a NIH thing. At least, certainly not any more than ActionCable is.
I see there is Ecto, Brunch, and ExAdmin for Phoenix, to cover our primary bases I spoke of, in Elixir/Phoenix.
For those that have switched to Phoenix, and have experience with these three- was it as smooth transition? What are the significant gaps, if any?
I've found this only really applies to CRUD-like, Rails-like projects, so about half of the projects I've built, so YMMV: it's maybe just me making the jump from Rails and expecting things to be more similar than they are.
Ecto is good; it's not quite an ORM, so there were a few WTFs when I tried to do things the same way, but on balance I prefer the Ecto approach; a Linq-like query language I like better than a [somewhat magic] ORM, the way it seperates concerns is good, and (in common with most of Phoenix) the way it works is pretty transparent. Not quite mature yet, though Ecto 2 seems to cover most of the functionality I found to be missing in 1.x.
I work primarily front-end,and I always had issues with the asset pipeline. I've found Brunch to be fantastic - there are always going to be a few issues when dealing with NPM, but other than that, I think they picked the absolute simplest JS-based task runner, and having direct access to the JS ecosystem is great. Brunch has been almost zero-config for me, has Just Worked with only a few `rm -rf node_modules`