Automatic Differentiation in Machine Learning: A Survey [pdf]
jmlr.org
jmlr.org
So, jets instead of numbers for differentiation, systems of boolean equations instead of numbers for answering some crypto questions... anyone know of other tricks in this vein?
Every external input to a program is assumed “tainted”. There are certain operations that make them “safe” (e.g. escaping a string makes it safe against SQL injection, etc.).
You simply attach this “tainted/safe” bit of information to each piece of data and flow it through your program. There is a simple calculus for combining these tainted/safe values, i.e. appending a tainted string to a safe string produces a tainted string, etc.
The goal is to prove that tainted values can’t make it to the business logic, so that your program is safe against certain kinds of malicious input.
[1]: https://en.m.wikipedia.org/wiki/Abstract_interpretation
https://en.wikipedia.org/wiki/Higher-order_differential_cryp...
[0]: https//news.ycombinator.com/item?id=18306860
* F. Wang, X. Wu, G. Essertel, J. Decker, T. Rompf, Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator, see [1], and [4] for a presentation based on this paper.
* S. Laue, M. Mitterreiter, J. Giesen, Computing Higher Order Derivatives of Matrix and Tensor Expressions, see [2], discussed in [3].
Quite a bit of exciting work in better understanding backpropagation going on right now.
[1] https://arxiv.org/abs/1803.10228
[2] http://www.matrixcalculus.org/matrixcalculus.pdf