Markov Chain Monte Carlo analysis of climate-change variables
nature.com
nature.com
And because there is likely no simple set gradients to follow, I’d suspect the Markov probabilities let you further navigate local maxima and minima more effectively. Something like a breadcrumb trail of past paths through the forest.
I may be missing the point though; I'm only versed in the mathematics.
I also wonder if differentiation doesn't have some nicer properties anyway, being a linear operation as well. Typically you can get quite far by showing things work for a sufficiently broad class of functions and just extending things from there. In particular you could use the Fourier transform, which trivially commutes with taking expectation values.
You can always try something abhorrent like sin(e^(-1/x^2)) or something to make things go wrong though.