I think both points are right. There's times when it's useful and desirable to be specific about lifetimes, and there's also times where it's annoying noise.
Bignum arithmetic is an example of the latter. You want to just work with numbers, and in Python you can, but in Rust you must clutter your code with lifetimes and borrows and clones.
Swift's plan to allow gradual, opt-in lifetime annotations seems really interesting, if it works.