Autodiff provides benefit since differentiation can be expensive numerically while easier symbolically. Integration is, in some ways, the opposite. Integrals with no symbolic representation can be estimated with quadrature methods.
While it's clear what you meant, it's mathematically important to note that functions like yours, and, indeed, all continuous functions, definitely do have indefinite integrals—that of `exp(x^2)` even has a name (https://mathworld.wolfram.com/Erfi.html); those integrals just aren't elementary, in the same technical sense in which you are using the word (https://en.wikipedia.org/wiki/Elementary_function).
Exp(x^2) is a finite composition of a polynomial and an exponential function so from what you posted, it does fall into the Wikipedia definition of elementary function.
They were arguing that the antiderivative F of f: x ↦ exp(x²) is not elementary, whereas f itself is elementary. But both F and f are well-defined functions, which means that f has an indefinite integral, it just happens to be not elementary.
Sometimes Markdown eats following parentheses and sometimes it doesn't, and I've never figured out why. When I first viewed the post it showed me the parenthesis not being eaten, but then it did it anyway when I viewed it from the main page (too late for me to edit). Thanks for fixing it up!