966 karma · joined April 1, 2008
To answer your actual question, I personally do it because I almost wish the journals would do something nasty to me. Then, I would tell everyone I meet at every conference about that for the next few years and would hasten the decline of the current status-quo.
I've used lots of operator overloading based autodiff packages for C++. They are great, but the issue is not how the function is recorded (I used operator overloading myself in my python package) but how it gets executed at runtime. Unless a compiler (or JIT) is called sometime between when the operator overloading happens and execution happens, the function is basically being interpreted at runtime. This is what happens in, e.g. ADOL-C, SACADE, and CPPAD, all of which come with a significant (e.g. 20x) performance penalty as compared with hand-written derivatives.
A[12800]=A[0] * A[6400];
A[12801]=A[1] * A[6480];
A[12802]=A[12800] + A[12801];
A[12803]=A[2] * A[6560];
A[12804]=A[12802] + A[12803];
A[12805]=A[3] * A[6640];
Which would be very easy to deal with. The only problem is that I sometimes call out to libraries for exp, log, sin, cos, etc.As to the multiple procedures: Well, as I was doing it, even a single procedure can be many hundreds of megabytes large. Even for very very simple code, however, I noticed that GCC was superlinear in code size. I suppose I could somewhat arbitrarily break up the code into arbitrary functions. I wonder if that would speed things up?
Other autodiff tools: Well, they basically trace execution, but then run an interpreter instead of trying to actual generate compiled code. I wanted to both be faster than that and have the wonderful experience of writing pure python...
My question is: could I leverage pypy somehow to avoid this? Can I output RPython? Can I output whatever RPython is compiled down to instead? Can I do this with no more than, say, a 3x penalty compared to c?
(I apologize for asking a question only marginally related to the particular article here...)
I come from a physics/cs background, and find that standard treatments of control are very much intended for EE folks. But this seems like a historical accident-- the techniques would seem to be very broadly useful in other fields.
(damage to society) <= (fine) * (risk of getting caught)