I tend to think of it in that way as well.
To me, "functional programming" taken back to origins in Lisp and the lambda calculus is a style of programming that focuses on the composition of functions.
The "purity" element is merely the consequence of this focus: in order to be composable, our functions largely need to be free of side effects, and take and return values as mathematical functions do.
I think the emphasis on function composition also makes for a better, more believable sales pitch, once you take into account first-class functions as values and the kinds of abstractions and patterns they allow. I have a version of this pitch which goes like this:
Imagine a function that, given the right inputs, could produce any output in the known universe. To which you might rightly answer "that's pure nonsense". The sheer volume of internal logic would naturally be more complicated than everything within that universe.
But if that function can itself receive other functions as inputs, then the potential outputs expand almost infinitely.
And indeed, this is why the lambda calculus itself has proved Turing-complete: simply by composing functions within functions in this way, you can literally recreate the whole of computing and mathematics.
In practice though, people have their limits, and the day to day operation of a computer does require occasionally breaking the perfect mathematical puzzle for the sake of getting work done, and the vast majority of languages in existence allow for exactly that. A definition that doesn't allow for them might well be "true" in the "pure" sense of the word, but it's not very useful for people other than mathematicians.
And hell, even the Turing-machine uses mutable state.