I tend to use the term "function-oriented" rather than "functional" since I think it fits the spirit of the phrase better. You're right that language features alone won't distinguish enough, and that means that functional programming is more about culture and standard practices. Once you embrace that, it's a bit easier to wrangle out a definition.
The definition I have in my head is something along the lines of "functions are the tool that the language prefers to address common software design concerns." So for example, Encapsulation is a big deal when designing software. Object-oriented languages will address this concern with their object system (via accessibility modifiers), while function-oriented languages tend to address this with functions, via lexical scoping and closures. OOP languages use their object system to achieve runtime-dynamic behavior through inheritance or composition or interfaces, while function-oriented languages will more often try to pass first-class functions as arguments. And so forth.
It's not a 100% thing. Common Lips will solve some of its problems with macros or its module system, and ML will solve some of its problems with types. But you do find the languages (or communities) prefer functions over their object systems, while e.g. Python or Ruby prefer the reverse.
Although, I do find myself wondering if Haskell should actually be considered a type-oriented language. Again, it's not just about the features of the type system, it's the fact that the type system is the main tool that it tries to use to solve every problem about designing software.