Setup Anaconda, Jupyter, and Rust | Hacker News Reader

Setup Anaconda, Jupyter, and Rust

datacrayon.com

71 pointsbatterylow22 comments

Worth mentioning that commercial use of anaconda repositories (including via miniconda, to my understanding) is no longer free and requires a commercial license.

https://www.anaconda.com/blog/anaconda-commercial-edition-fa...

Beware if you are following this guide for your work.

> We clarified our definition of commercial usage in our Terms of Service in an update on Sept. 30, 2020. The new language states that use by individual hobbyists, students, universities, non-profit organizations, or businesses with less than 200 employees is allowed, and all other usage is considered commercial and thus requires a business relationship with Anaconda.

Looks like small companies are exonerated from commercial license.

> businesses with less than 200 employees is allowed

This is great!

No businesses with >200 employees should be allowed to exist.

I don't know about that... I just meant that it's exceptionally reasonable

why? seems like that would make any large scale business venture completely impossible and would cut us off from important economies of scale

How the F#@! would you replicate the capabilities of say lockheed martin or something with a thousand tiny companies on a massive top secret plane project. That's absolutely ridiculous.

From the banner for "Data Analysis with Rust Notebooks" this looks someone realised that both data stuff and Rust are popular on their own and decided to put them together to sell an e-book. It's cool that this is at all possible but ultimately it forces Rust into a use case that it isn't well suited for. I can't see a good reason to pick it over Python/R/Julia.

Speed, static types. Definitely not a general alternative.

In the section before there is agreement with your comment:

> Can we write and execute all our code in a Jupyter Notebook? Yes! Should we? Probably not. However, I enjoy the workflow, and making this an enjoyable process is important to me.

Yeah I think Rust's place in this ecosystem is a C/C++ replacement. All the Python and R data science packages call into native code to do the heavy lifting--the linear algebra, gradient descent etc. The next Tensorflow or PyTorch type framework could be implemented in Rust instead of C++. But it shouldn't matter to the end users--they would still use the Python or R bindings so they can have an interactive REPL environment to do their analysis in.

Julia is interesting because it's performant enough to implement these things directly--Julia's machine learning frameworks are written in pure Julia (with the exception of the calls to CUDA libraries for GPU) and can achieve near-native performance, so solving the "two language problem" is part of its value proposition. I do wonder how valuable it is for the whole stack to be written in one language, and whether that will blur the lines between the software engineers who implement data science packages and the data scientists who use them.

Julia can do GPU kernels (Intel, AMD, Nvidia) also, they just aren't all implemented in pure Julia (or tuned) yet, hence the CUDA lib calls.

https://juliagpu.org/ https://juliagpu.org/2020-09-28-gemmkernels/

The decision to shoehorn rust into a jupyter notebook is baffling, especially for something like data science. I like rust and all but come on. Just because it’s possible doesn’t mean you should.

Why Rust? Conda and Jupyter are python based. What does one hope to achieve with a setup like this?

Rust is a great language for writing Python extensions, using a library like PyO3. Some of the most CPU-intensive parts of my company's feature engineering pipeline are now handled in Rust.

I would enjoy writing Rust for Python even more if I could compile and run Rust straight from the same Jupyter notebooks that I prototype my Python code with.

https://github.com/PyO3/pyo3

I wasn't aware of pyo3 thanks for that.

conda, although written in Python, is language (and somewhat OS) agnostic. We provide lots of C, C++, R and Rust packages as well as Python ones.

I was initially confused over the name conflict with the GUI Linux distro installer Anacoda: https://en.wikipedia.org/wiki/Anaconda_(installer)

In the context of this post, Anaconda is a Python and R distribution using the conda package manager: https://en.wikipedia.org/wiki/Anaconda_(Python_distribution)

The python and R distribution is much better known than the GUI Linux installer.

Depends on the bubble you are in.

> Once Miniconda is installed, we need to create and configure our environment. If you added Miniconda to your PATH environment during the installation process, then you can run these commands directly from Terminal, Powershell, or CMD.

AFAIK, the PATH modification route is no longer recommended. Instead one should use, eg `. miniconda3/etc/profile.d/conda.sh` to add a particular miniconda installaiton to the environment.

It does say "if", but with MiniConda's CLI installer it's currently

> Do you wish the installer to initialize Miniconda3 by running conda init [yes|no]:

with the default as [no] as well

Reply on news.ycombinator.com