Because many system package managers doesn't support multiple version of a same package easily. Also, there are system (used to) without a package manager.

> causes python devs to often be way too specific in version specs

I don't think anything could fix that, because it is totally up to library developers to decide whether they want to make breaking change or not. So being able to use different exact version of dependencies for different projects are important.

BTW, I find it annoying that many distro package manager doesn't make it easy for user to install package for themselves without root. Even Nix requires you to create /nix IIRC.