Sorry, I was out.
Ok,
f(x) = cuberoot(x^3)
f(0 + eps) = cuberoot((0 + eps)^3) = cuberoot(eps^3) = eps
e.g. value is 0 and the slope is 1.
Ok,
f(x) = cuberoot(x^3)
f(0 + eps) = cuberoot((0 + eps)^3) = cuberoot(eps^3) = eps
e.g. value is 0 and the slope is 1.
>>> import autograd.numpy as np
>>> from autograd import grad
>>> def fn(x):
... return np.power(np.power(x, 3), 1/3)
...
>>> gradfn = grad(fn)
>>> gradfn(0.0)
/usr/local/lib/python3.6/dist-packages/autograd/numpy/numpy_vjps.py:59:RuntimeWarning: divide by zero encountered in double_scalars
lambda ans, x, y : unbroadcast_f(x, lambda g: g * y * x ** anp.where(y, y - 1, 1.)),
/usr/local/lib/python3.6/dist-packages/autograd/numpy/numpy_vjps.py:59: RuntimeWarning: invalid value encountered in double_scalars
lambda ans, x, y : unbroadcast_f(x, lambda g: g * y * x ** anp.where(y, y - 1, 1.)),
nan
>>> import torch
>>> x = torch.tensor(0.0, requires_grad=True)
>>> y = ((x**3) ** (1/3))
>>> y.backward()
>>> x.grad
tensor(nan)
> a complicated representation of) y=x.
The problem started as that. So no surprise.
Sure, we call that computer algebra. As soon as you start doing that, you aren't doing automatic differentiation, you are doing (at least in part) symbolic differentiation.
#include <math.h>
#include <stdio.h>
#include <stdlib.h>
double f(const double x) {
return pow(x, 1000) / pow(x, 998);
}
int main(int argc, char* argv[]) {
if(argc != 2) {
fprintf(stderr, "Usage: %s x\n", argv[0]);
exit(1);
}
const double x = atof(argv[1]); // demo code without error checking
printf("x = %g, f(x) = %g\n", x, f(x));
exit(0);
}
gcc -Wall -g -o test test.c -lm
petschge@localhost:~$ ./test 0
x = 0, f(x) = -nan
petschge@localhost:~$ ./test 1
x = 1, f(x) = 1
petschge@localhost:~$ ./test 2
x = 2, f(x) = 4
gcc -Wall -O3 -ffast-math -g -o test2 test.c -lm
petschge@localhost:~$ ./test2 0
x = 0, f(x) = 0