Hacker News Reader

Comment by tome | Hacker News Reader

I don't find this very convincing. Continuations are well known to be completely mystifying. I wrote up what I believe to be a much simpler reverse mode AD transformation that uses only basic concepts:

http://h2.jaguarpaw.co.uk/posts/automatic-differentiation-wo...

View on HN

Your notes, while simple, don't cover many of the things the paper does, such as binding or functions or sharing.

You're right they don't cover functions. I don't understand what about binding or sharing they don't deal with though. Could you elaborate?

For starters, what if you want to differentiate through code that traverses a collection, for example:

    val x = ...
    ys.map(y => y + x)

Each loop iteration needs to contribute a gradient update to x, which is defined in an outer scope. And what if y => y + x is not given as an inline lambda, but defined elsewhere. It doesn't seem like your blog posts discusses any such cases.

Correct, it doesn't discuss that case, indeed it doesn't treat user-defined functions at all. I just don't understand in what sense it doesn't treat binding or sharing.

> Continuations are well known to be completely mystifying.

Could you elaborate on what leads you to think this?

I don't believe that continuations are a corner of programming language theory that is well-loved and and well-understood by programmers at large, are they? Even amongst experienced functional programmers they seem to be thought of as rather difficult. Oleg Kiselyov has written extensively on them but his work has not been brought to the masses in the same way that his work in iteratees has.

http://okmij.org/ftp/continuations/index.html

Ahhh! Right! See, the context here is important. Continuations are pretty widely known and important in the context of programming language implementation and theory, which is why I was confused by your comment. In the context where this paper was presented (to programming language researchers), making the link between delimited continuations _would_ demystify a bunch, and could allow for transferring of knowledge and techniques between the fields.

Having looked again, the authors do explain why they believe that continuations (as opposed to simply higher-order functions) are necessary. See my other comment at https://news.ycombinator.com/item?id=22347428.

Reply on news.ycombinator.com