Python vs. Rust for Neural Networks

At the same time, comparatively tedious languages like Rust will never attract data science practitioners. They don't care about the kind of safety it brings, they don't care about improving performance in a component that's idle 99% of the time.

The bulk of the load in an DL workflow is CUDA code and sits on the GPU. Even the intermediate libraries like cublas would only see marginal-to-none benefits of being reimplemented in rust.

This is a cool project, but it has no chance to displace or even complement Python in the data science space.

Calling Python code from Rust or Rust from Python is totally doable, and there is in my view no reason why you shouldn't use both in the use cases that suit them.

And the speed part is serious. Some guy once asked for the fastest tokenizer in any given language and my naive implementation came second place in his benchmark right after an stripped down and optimized C variant.

So using Rust for speed critical modules and interfacing them from easy to use Python libraries isn't exactly irrational.

Python is an interface to C, C++, and FORTRAN for a lot of stuff. There are even crossover libs for running R.

This is like comparing apples and steaks.

Also, does Rust have a GPU/CUDA backend yet?

Rust seems great, to be honest, just not universally so. Nothing wrong with defining the boundaries.

I’m not totally convinced rust knows what it’s aiming to be, other than replacing c++...

The only downside i've seen is sometimes the programmer will want more safety. In such a scenario Rust for the "frontend" would be very useful.

So we have two concerns, frontend and backend. For the backend Rust would perfectly acceptable, but i'm not sure it is fixing a safety issue/etc in other (C/etc) languages - aka, perhaps little value in the backend. For the frontend it only has value in some areas.

Regardless, i love Rust and would totally welcome any tooling to keep me in Rust. However i'm not an ML person hah.

But for neural networks, people often prefer to use special hardware like graphics cards, since graphics cards are really good at doing relatively simple math on many pieces of data at once. So they create special libraries like TensorFlow that can send commands to the graphics card instead of doing the math on the CPU. (And they don't use Numpy because even though it's highly optimized, it's highly optimized for CPUs, and graphics cards are a lot faster than CPUs at running neural networks.)

If I'm not doing research on new methods but want to build a model for a particular problem using well-known best practices, then all the custom code that my app needs and what I need to write is about the transformation and representation and structure of my particular dataset and task; but things like, for example, optimized backpropagation for a stack of bidirectional LSTM layers are not custom for my app, they're generic - why would I need or want to reimplement them except as a learning exercise?

That'd be like reinventing a bicycle, for generic things like that I'd want to call a library where that code is well-tested and well-optimized (including for GPU usage) by someone else, and that library isn't numpy. Numpy works at the granularity of matrix multiplication ops, but applied ML works at the granularity of whole layers such as self-attention or LSTM or CNN; which perhaps are not that complex conceptually, but do require some attention to implement properly in an optimized way; you can implement them in numpy but you probably shouldn't (unless as a learning exercise).

1. Numpy doesn't run on GPU.

2. Numpy isn't high level enough for NN building.

3. Numpy doesn't have auto differentiation.

Other options solve some of these - Autograd solves 3, Jax solves 1 and 3, etc.

But if you want all 3 then you want to use Pytorch.

They didn't. At the end of the article they discuss this.

"In fact it’s worse than that. One of the exercises in the book is to rewrite the Python code to use vectorized matrix multiplication. In this approach the backpropagation for all of the samples in each mini-batch happens in a single set of vectorized matrix multiplication operations. This requires the ability to matrix multiplication between 3D and 2D arrays. Since each matrix multiplication operation happens using a larger amount of data than the non-vectorized case, OpenBLAS is able to more efficiently occupy CPU caches and registers, ultimately better using the available CPU resources on my laptop. The rewritten Python version ends up faster than the Rust version, again by a factor of two or so."

The original python code was written to be easily understood; not performant. Shifting more of the work to the libraries improved the performance.

Thanks to ML.NET, I can enjoy .NET JIT/AOT compilers performance, while binding to the same Tensorflow C++ libraries that are wrapped for Python.

Swift and Kotlin enjoy similar bindings on their respective mobile platforms.

Then there is Julia.

Python might have gotten there first, but its kingdom is already getting slowly eroded.

Python is an ok interface language, in that it's script-like, dynamically typed and simple to comprehend. It's popular, which makes on-boarding efficient due to the sheer volume of tutorials online. And, it has built up a large ecosystem, because of the last two points.

That said, it's naive to suppose that python is the currently-ideal or future-ideal interface language. It's just ok, plus it's popular.