José Valim Reveals “Project Nx” (Numerical Elixir) [audio]
thinkingelixir.com
thinkingelixir.com
We are going to make the repository open source next week and, for those who haven't listened to the podcast, I want to align everyone's expectations and say it is very early material. Expect bugs, breaking changes, etc.
On the other hand, it is also a great time for anyone who wants to jump in and help shape things, so feedback will be very welcome!
I think the elixir/erlang way of thinking about "concurrency as a special case of distribution" will apply to Nx, except "gpu as a special case of distribution" which means there won't be a leaky abstraction for remote gpu. I tried to get the Julia folks to adopt this sort of an attitude towards the juliagpu, but I'm a bit of a nobody so it didn't take.
Very excited! And I'm whipping up a little bit of something interesting related to Nx myself that hopefully I can release around the time of your talk, but I don't want to make any promises yet.
There is more than one difference between GraphQL and REST, could you clarify this analogy? Thanks
Here’s something I’m confused about. For fast numerical code, it’s vey important to be disciplined about memory allocation. So in addition to considering of immutability -vs- mutable pre-allocation, it would seem that the scheduling model ought to be substantially different. Instead of constantly switching between concurrent processes to ensure none of them monopolize the processor, would it make more sense to finish one computation while the contents are in memory/cache, before cleaning it up and swapping to another process? (Especially for really heavy computations). Will process scheduling have to be micromanaged? Is the BEAM compatible with this mode of usage, or is this maybe not a concern?
BEAM concurrency only involves memory allocation when the processes are created. Switching between them doesn't involve allocation and is very quick (don't forget we are talking about BEAM processes here not OS ones - it's all in userspace and very efficient). As long as you pre-create your pool of worker processes concurrency doesn't incur much allocation hit.
You're right that immutability might do though. I am curious to see how this goes - the speed that Elixir normally runs at combined with some data sharing optimisations means that this issue doesn't exist normally. At the speed we care about when doing numerical work it's more of a concern. Or maybe we'll never get into the realms of full optimised C++ where things are sized for cache lines etc. (and that's ok).
I've always dreamt that someday you might be able to reach for Elixir and get everything you needed for an entire stack of a company and it seems to be getting closer and closer. The building blocks for a web app have been around with Phoenix and Ecto for quite some time. LiveView was introduced to make more user friendly frontends. GenStage, Flow, and Broadway introduced ways to run data processing pipelines. And now we're seeing the initial steps to potentially have ML capabilities.
Remember, Java and the JVM essentially have this today: front end frameworks, back end frameworks, data pipelines, big data processing, entire databases, etc.
In your opinion, what will make the Elixir flavor of this situation different?
More importantly, the functional style used with Elixir will likely mean shallow usage patterns that are more composable than the deep layers upon layers of 'architecture' that plagues long lived class-based OO designs.
I've written Rails professionally for almost 10 years and have been following / playing with Elixir since 2016. While Ruby / Rails are amazing and the ecosystem is well suited getting a project off the ground extremely quickly, I'm just a big Elixir fan and hope to work with it professionally one day. I'll parrot reasoning from some of the sister comments.
"having an entire environment of consistent actors, fault tolerance, hot reloading, etc is a tempting proposition." - Very tempting indeed!
"Code elegance and development joy, maybe?" - Coming from Ruby / Rails, I agree 100%
"Not being statically typed" - Again, I've worked mainly in dynamically typed languages. I've experienced typing in Javascript and wasn't the biggest fan. But don't get me wrong, Rails can be awful when trying to dig multiple objects deep to understand what the data structure of the return value is so its not completely off the hook.
I think Elixir's big selling point is getting the BEAM runtime wrapped in a more modern language with more modern tooling. Lots of the positives people list about Elixir are due to BEAM, which you can also get in Erlang.
I was in love with Ruby and hated Java around 2009, but the latter has gotten so much better with IntelliJ IDEA, to the point where I have to admit that working with JavaFX from IntelliJ was likely my most serene programming job ever. To me, the whole ecosystem is only worthwhile because of JetBrains.
Robust IDE support for Elixir is probably a must-have requirement for many nowadays. I am really tempted by the design of Elixir and find Java/Go/C++ aesthetically repulsive, but I'll happily forget about that if I can keep my refactoring muscle memory.
It was very interesting experience being a long-time Pythonista having to be like "Isn't this awesome language with a Ruby-like syntax great???" and being told nah, let's just use async python by people who had spent years writing Ruby code.
This also helps with recruiting, as the tools work similarly. I didn't spend a lot of time in Rails, and decided not to go that way again, but I did appreciate that I got to recycle some of that new thinking when I started writing Node.
Phoenix also seems to be designed to recruit Rails developers. You could do worse than porting missing tools and crates from Ruby, and as far as I'm aware nobody has nominated themselves as the TJ of Elixir.
In terms of what it's good at, we very successfully employed Erlang as the command and control language for some semi-embedded system medical devices. The runtime is predictable in terms of performance and memory, and fast enough for a lot of things. Where it wasn't, it's easy enough to talk to C. All the fault-tolerant bits are nice for a system like that where you want it to take note of a failure and restart the subsystem that failed. Lots of good logging, introspection and diagnostic tools exist.
AFAIK, any modernish language is good enough to get something up and running regardless of the task.
In the middle you will probably have a large number of problems that it's ok-ish at. But not really sure on what the distribution is here.
Can you give an examples? Respectfully, you were asked for tasks and instead gave a description of the category.
I can only think of real-time, or real performance limited software (example: rocket guidance software, aviation, etc). With these every modicum of performance per millisecond matters; but these constraints only apply in a few (relatively) small fields of software development.
Of course, maybe there's a generalized BEAM to C call library I'm not aware of? (I didn't think to search for one until now)
Also, honestly, it's a little easier to have a C daemon than an erlang daemon. Just call daemon() before the service loop vs setting up an app file. Of course, you have to build hotload yourself from dlopen and friends, and you lose out on a lot of features.
I'm looking for the equivalent of Perl's h2xs, or something like that. (which incidentally, maybe I should be writing these things in perl ;)
Elixir releases are built in and easy to use. You don't have to write an app file directly for instance.
I usually use ports for C interface (rather than NIFs). I have a little boilerplate C and Elixir that I recycle to do the ground work and then it's just your specific implementation detail.
In practice I haven't found the immutability of Elixir that useful for concurrency because data is deep copied when it goes across processes anyways. This is in comparison to Clojure where you can safely and perfomantly share immutable data across multiple threads.
To be clear, for NPCs it worked fine. But players are hoarders and in an RPG that can result in a ridiculous amount of data being attached to them.
A lot of languages that have more limited existing use have more gaps in their ecosystems. That means that to get something going quickly, you run into cases where you spend a lot more time building (or validating and tweaking existing but relatively rough) support infrastructure that you would just grab a well-tested, established library for in a language with a richer community and ecosystem. This is an area where the top tier ecosystems (Python, .NET, JVM) really shine over even second-tier popular ones like Ruby, and where both of those tower over things like BEAM, which has the additional problem that it has a fairly high impedance for incorporating C libraries, where many other platforms (some also with stronger ecosystems of their own) also more easily consume C libraries.
This extends both to broad application domains (e.g., scientific libraries) and to things like probability of having an idiomatic, language specific, well-supported client for a new service you want to tie in and incorporate.
If you're running a server—sure, BEAM me up.
Things I like using it for:
- long running services
- doing anything w/ concurrency
- setting up cron-like processes that run every so often
- actor model anything
- stateful services
- web development using Phoenix & Ecto
- parsing
- encoding/decoding
Things I don't like are primarily related to it being a small enough community. That means libraries for third party services aren't always the best, and using it in the enterprise means that the smaller talent pool is less attractive when the company's priority is to use proven/boring tech for which you can hire tons of developers. I also don't think Elixir has a clear value proposition when you're writing small, single purpose functions for a serverless architecture (even though I really like writing it), or when you're company decides to go all in on Kafka for communication between micro services.
Just my two cents.
The single unix process that BEAM uses is more conducive to some kinds of systems than some of the hackier-feeling solutions for the Rails ecosystem to tie different pieces together.
What's the issue with Kafka and Elixir?
Being able to utilize erlang libraries means there are a ton of heavily production tested libraries you can easily drop into an elixir project.
I also encountered some pain with the AWS client for Elixir when our company implemented SSO. There was a lack of support for a certain flavor of environment variable if I recall correctly. It's not horrible, just a pain.
hard disagree on all but c++, .net and maybe python
I think people forget erlang has been around a long long time and I've written ruby for years and can speak with great confidence that the equivalent ruby libraries in elixir are typically much higher quality
In case someone here wants to read rather than listen to all the details, I wrote down some notes over at: https://elixirforum.com/t/anyone-who-wants-to-speculate-abou...
> Nx = Project for a collection of libraries. Nx is the core library, the other libraries depend on this core library
> If you come from Python, it can be though of as kind of like Numpy. Long way to go but working on that.
> “Slowness in Elixir” due to immutability and copying etc. Performance comes from module that is part of Nx called Numerical Definitions
> In the definition you write with a subset of Elixir. Elixir kernel is replaced with Numerical kernel.
> Based on Google XLA (accelerated linear algebra)
> You can give it a computation graph and exla compiles it to run efficiently on CPU or GPU
That's actually one of the things I am really looking forward to: see what other compilers people will integrate with. I am aware some of the neural network libs have pluggable compilers but it will be interesting to see it done at the tensor level. :)
As the founder of the language, the lead evangelist, and the main mouthpiece, yes, 'single handedly' may be hyperbolic, but only slightly. It's not to dismiss others' inputs, but it is to highlight how -insanely- engaged he is. I can think of no other language founder who has put in nearly the same legwork.
I don't pay particularly close attention these days, but I'd think Rich Hickey's involvement in Clojure over time, at least to the point Elixir is today, would be comparable.
Anecdotally speaking, he hasn't been hands on with Oban[1], yet he still offers advice and guidance around the project because it is in the Elixir community.
But I agree Jose is always helpful and the strongest pillar in the community.
But after listening to the talk, I went from thinking that Elixir was not a good fit to ML to actually excited with the future. Elixir's pipes are definitely a beautiful and concise way to define transformations, and we already have nice examples of them with Streams and Ecto.Multi that even make it surprisingly nice to debug the readable "tape", and being immutable itself allows it to nicely avoid the hardest parts in translating the language into an execution graph both for the creators of the library and for the user. Modifying the if macro to embed the conditional within the "tape" might be a little too much magic though.
Obviously this being on the beam you’d get the ability to run these actors across different computers fairly easily, hopefully scaling quite well.
You could also build FDTD systems like this and even acoustics simulations. I wonder if this is something the authors here are thinking about?
- It's slow, really slow, and memory heavy. All communication between processes involves copying and context switch.
- It's easy to introduce deadlocks (A calls B calls C calls A - now system is stuck) or unbounded message queue growth (if you use casts rather than calls)
- Everything is a single unit of fault tolerance (if one cell crashes, the entire simulation should be re-initialized from a known state) so all the processes need to be linked together in a massive web, which more or less defeats the point of process isolation.
I think it might make sense in the abstract actor model, it just doesn't map cleanly to Erlang processes.
> “Hey, what did you do over Christmas?”
> me: “I drank too much and tried to read a book”
> Jose Valim: “I rewrote my language’s kernel to afford tensorflow-like operations for efficient ML operations”
I had spent a little time trying to contribute to Tensorflex which was a google summer of code project for TF in elixir but life got too busy to proceed and then I didn't see any movement there so just figured the idea of ML was on pause and outside of Matrex numerics were also on pause. Happy to see it was still going on behind the scenes.
1. Work on modernizing erlport can still happen and be orthogonal to this
2. Moving data in and out of the GPU is expensive, sending the data to another process so it can copy it to the GPU would make us pay the penalty twice (and then again on the way back)
3. And even if you want to go ahead with 2, I would do it over the wire so I don't couple my nodes running Erlang/Elixir to the nodes running the Python stuff (because they may require different specs/scale)
4. We don't need people to move away from their computation frameworks, I will be happy if people start considering building new things on top of this one in the future :)
Elixir’s documentation is so good, however, I would expect most questions will be answered by the documentation directly.
It's like with trademarks: you can have more than one thing with the same name and not have a problem. If it was another build tool that helped you deal with your git repos that would be a real issue, but it's not... it's aimed at a totally different audience for a totally different reason.
This town is absolutely big enough for the both of them.
Or do you think other people have problems because when they search for, say, Kafka, or Jekyll, they keep finding references to literature instead of the tech they're using?
/s
I think the fact that a popular github repo (https://github.com/nrwl/nx), a domain (https://nx.dev/), and loads of blog posts and tutorials exist for the current NX is enough reason to pick something else. Just thinking off the top... how about NeX.
I use nx.dev and hope they see it the same way.
Picking a (short, memorable) name without colliding with some obscure open source project is neigh-impossible.
Take, for example, the web world. I believe Phoenix is technically the best framework out there, or at least on par with Rails. And it has not taken the web world by storm (yet). Why would Nx be a success without the support of Google or someone that big?
1) python really does not have broad cultural discipline around testing. Especially not in the ml field.
2) deploying python is not the best.
3) unlike TF, torch is actually not hard to understand and the code should be easily movable and portable out of it.
Small time ML users will care about 1&2. 3 makes it relatively painless.
Most importantly it may allow Elixir developers to not switch to C or Rust for intensive computations (hopefully).