This keeps being repeated and repeated and it is just a myth.
Even without a package manager adding dependencies manually is quite easy for the vast majority of popular libraries. With a manager it is the same as cargo.
I assume you are talking about old, broken, unsupported, closed source, binary only libraries, but then it is not a meaningful comparison.