Having a list of what version you're using of a single file library seems like an easy problem to solve. If nothing else you could put the version number in the file name and in a debug build print off the file name.
Let's say you depend on foo, which depends on 10 other libraries including bar, all of which depend on a library called baz. Then one day someone discovers an exploit for baz.
With npm, you only have one version of baz installed, and can easily check if it's a vulnerable version.
With single-file libraries, baz was built into a single file. Then bar was built into a single file containing baz. Then foo was built into a single file containing bar and other libraries, all which included baz.
Now your library contains baz 10 times, all of which might be at different versions, and none of which you can easily check. (You can check your version of foo easily enough, but baz is the library with the exploit)
You are talking about your own thing. Everyone else is talking about the number of files, not the distribution and update mechanism.
The packaging system could support single file and still be able to track versions and upgrades. The JVM ecosystem is effectively single file for many deps esp now that jars can contain jars.
How is that not talking about PIP?
That being said in C and C++ the single file libraries typically have no dependencies, which is one of the major benefits.
Dependencies are always something that a programmer should be completely clear about, introducing even one new dependency matters and needs to be thought through.
If someone is blasting in new dependencies that themselves have new dependencies and some of these overlap or are circular, and nothing is being baked down to standing alone it's going to end in heart break and disaster no matter what. That basically the opposite of modularity, a web of dependencies that can't be untangled. This applies to libraries and it applies on a smaller scale to things like classes.
Any dependency needs to be considered, there is no way around that. There is no reason to make it more painful just to make a programmer's life more difficult and single file libraries, especially those that have no dependencies themselves are the best case scenario.