Hacker News Reader

I've been really struggling to see the benefit from autodiff... I work with DAGs, where the data processes are batches that could be processed in parallel and I would love to speed that up, or to enable some gradient descent style optimization.... but I can't figure out how autodiff plays into it. Do you know of a good place to help learn how it could apply to my domain?

View on HN

Firstly vmap() might make it easier for you from a programming standpoint. You can think of operations in terms of single instances and then autobatch them.

Also, jit() is where you'll likely see your speed increases.

As for autodiff - not every application will use it. What exactly are you trying to optimize? If you can parameterize your model somehow (e.g. parameterize edge weights) and then figure out some way to measure how "bad" your model is, you can use autodiff to tune your edge weights to minimize that metric. Not too familiar with DAGs but the first step is figuring out how to parameterize your model in a way that tuning the params can lead to your goal (and how to measure how close you are to that goal)

Reply on news.ycombinator.com