There are some situations where having some animation can "attract attention" to parts of the UI, for guidance ("there's some info text here"), feedback ("operation complete"), state changes, timing. All this of course doesn't apply to decorative animations.
With that said: a lot of the situations I mention above are manufactured. They are often because of changes happening away from the mouse, or because the interface is brittle or too slow, and the user doesn't have confidence that something really happened, or because the organization of elements is not functional and things are too far apart that the user might miss something in a totally different part of the screen.
IMO, with enough thinking you can come up with alternative interfaces that don't require animations at all.
Basically, animation provides additional information about object state. Removing that extra information increases cognitive load.
This isn't to say all animations are useful. Many animations are excessive or completely unnecessary, which is probably what has given you a negative view of them.
Also, as an example just try to use a window manager that just switches instantly to a new desktop, vs a very short animation. I find the former easily disorienting, and it is even more so when you have a more direct gesture controlling the action (e.g. touch or mousepad). Of course there is too much of everything, and too much animation sucks, no question. So is too much fat, but that doesn't make normal amounts any bad.