Actually, because differentiation is a limiting process, this relies on the bounded convergence theorem or monotone convergence theorem. However, I imagine this would hold in most distributions under consideration.
I also wonder if differentiation doesn't have some nicer properties anyway, being a linear operation as well. Typically you can get quite far by showing things work for a sufficiently broad class of functions and just extending things from there. In particular you could use the Fourier transform, which trivially commutes with taking expectation values.
You can always try something abhorrent like sin(e^(-1/x^2)) or something to make things go wrong though.