Differentiable programming for gradient-based machine learning | Hacker News Reader

Differentiable programming for gradient-based machine learning

forums.swift.org

34 pointsundefinednull16 comments

Has anyone got experience in using other things than python for machine learning? What are the rivals out there?

In my experience: Julia, Java, C++. Lack of libraries in a pain; OTOH the quality of most libraries (in general) tends to be poor.

Examples of library issues: there is C++ support in tensor flow, but not for training.

Agree to that completely. It is also hard for the libraries to spend their energy / time in building for languages that covers only a few percentage of total users. :(

For anything involving differential equations, Julia is way better than anything else. I also wouldn't be at all surprised if Julia catches up to python for way more machine learning within the next year

Julia doesn't lack libraries and the quality imo is pretty good. Did you look at PyCall.jl?

Interesting! I did not yet! will take a look. I remember swift was working on something similar to seamlessly call Python.

Particularly outside deep learning, R remains popular especially for those closer to the stats world.

Matlab and Mathematica both have interesting machine learning features but are probably orders of magnitude lower usage levels compared to Python.

I'm using Julia instead of Python for NLP and it's great. If I need a python library I just use PyCall and so far it has worked flawlessly.

Can you say a bit about the advantages of each? I'm just starting to learn deep learning and planning to use it for NLP. I'm taking the python-based fast.ai course. But I'm tempted by the elegance of Julia.

If you're starting you should use Python and when you have a more solid grasp you can switch to Julia. That's mostly due to the fact that there are much more tutorials and resources for dl in python.

There is a rather active community for Go in data science and machine learning. +1700 members on the #data-science channel on the Gopher slack, a few books on the topic, and a few libraries:

Books

Machine learning with Go - https://www.amazon.com/Machine-Learning-Classification-Time-...

Go Machine Learning Projects - https://www.amazon.com/Go-Machine-Learning-Projects-Ultimate...

Libraries

https://gorgonia.org/

https://www.gonum.org/

A good talk (although from a from years back), explaining why Go for ML could be a good idea:

https://www.youtube.com/watch?v=D5tDubyXLrQ

Is anyone still excited about differentiable programming? At least in NLP it seems like a lot of the energy has shifted to large scale overparameterized models like BERT, e.g. you don't need arbitrary control flow in your model, all you need is attention.

The differentiable programming framing is quite useful in the scientific simulation space, because it lets you couple traditional physics simulations to either neural surrogates or some sort of bayesian model. Enough of it is happening in the Julia world that were very actively improving the compiler to support it.

Here's a stunning recent example from computer graphics: http://rgl.epfl.ch/publications/NimierDavidVicini2019Mitsuba...

Attention is cool but not to the level of higher-order cognitive architecture. Further, there are many uses of differentiable programming besides AI

This talk by Eric Meijer got me intrigued of the subject ... (Based on this talk, I think I almost understood it for an hour or two):

"The principles behind Differentiable Programming - Erik Meijer" https://www.youtube.com/watch?v=lk0PhtSHE38

Reply on news.ycombinator.com