Hacker News Reader

Comment by davidw | Hacker News Reader

If I may ask, what are you working on where Elixir hits the sweet spot compared to other technologies?

A news aggregator (and premium news chatbot) that indexes and analyses around ~150.000 new articles a day (http://im.fo)

I'm absolutely certain the real time processing would be unfeasible in any other technology in terms of complexity and the minimal compute resources it's running on.

Modules like broadway, ash, oban, phoenix liveview ... make it not just a pleasure to work with but insanely performant.

With over 20 years of programming experience, I can say with certainty that there is no language that makes me as productive as elixir. It's at least 10x my python productivity (despite being at an expert level in python as well).

You know that Elixir is on the low end of performance right, so you take Go/Java/C# that are close to 10x faster.

I don’t know enough about news aggregators to evaluate the claim, but presumably that is why parent mentioned both complexity and compute, not just performance.

Which parts of his application need a 10x speed improvement?

performance has many dimensions. for example the cost/speed of spawning a process/thread and their intra process communication.

Very cheap in C# and Go. I assume Java has now closed the gap with its Green Threads implementation.

(Spawning an asynchronously yielding C# task is ~100B of allocations depending on state machine box size, with very small overall overhead and threadpool handling millions of them, they are cheaper than Elixir tasks which make different tradeoffs (and are subject to BEAM limitations), you can try this out on your machine by running the examples from this thread: https://news.ycombinator.com/item?id=40435220)

thanks for sharing!

For "straight line" single thread number-crunching, other languages will often be faster. With the new JIT, I doubt they are 10 times faster, but you're right there is a difference.

That's often not the limiting factor though. Elixir makes it very easy to have excellent parallelism on your work so you actually take full advantage of processor. The design of the BEAM means that things are naturally quite low latency, and often you lose performance due to just waiting for things (this characteristic is why webservers on the BEAM are pretty fast).

The other key aspect is NX - like ML libraries in Python, Elixir is just orchestrating and the number crunching is done in C libraries or on GPU etc.

If you need per-core number crunching performance you'd reach for Nx, similar to how you would do the crunching in Python. With OTP it's then (almost) trivially concurrent.

Compared to squeezing performance out of multiple cores with the JVM it's absurdly convenient and consumes way less RAM. I have two reasons for still working with the JVM, multiplatform desktop GUI and high quality PDF libraries that support rather low-level aspects of the standard that I need. It's kind of obvious why these things aren't readily available on the BEAM, though.

Reply on news.ycombinator.com