The problem with it -- and there IS a problem -- is really a problem of applicability. Excel, like Lotus before it, is the first place many people encounter the ability to create their own logical conditions.
For a huge subset of these people, it's also the LAST such tool they learn. And now they have a hammer, so everything is a nail. I don't just mean finance people who live and die by spreadsheets; I've met engineers who would solve problems in Excel macros that would've been better attacked in perl or Python.
The other "problem" is the degree to which a horrifying number of organizations end up depending on very, very complex spreadsheets full of arcane and undocumented formulas and macros, and for which change control is "save-as."
But, again, none of this is a problem with the product itself, or with the idea of spreadsheets.