The real problem is that all of the major package managers require root, and usually you can't set the install prefix to a different folder than '/' (like for example '~/.local/'), without using chroot (again root is needed there).
It is true that some packages DO actually require root (like obviously the kernel), but most really don't.
A package manager that works like this would be portable to any distro. I think Nix and Guix are supposed to do this.