This is both misleading and false. Functions, by their very nature, "do something."
Perhaps you're referring to the immutability of data in a functional language? If so, say so.
"We don't manipulate the syntax tree, but it doesn't mean it isn't there."
A statement so ambiguous as to be meaningless. All compiled languages have a "syntax tree" lurking in the background. It just so happens that Lisp and its variants let you manipulate that tree directly, rather than writing in code that gets converted to that tree at compile time.
Perhaps you're referring to the fact that in Haskell, you often don't control the order in which code is executed, unlike in an imperative language where you have explicit control?
"...just saying that functions belong to the bucket of mathematical objects. As are integers. And sets."
Nope. The quote you use has it right. The names of functions are variables, whose value is looked up like any variable. This lets you pass functions as arguments to other functions, or use functions as the return value for other functions. Functions are most definitely not just like integers in a functional language.