I personally use Nixpkgs to fully isolate python installation on per project basis. And inside this isolated env, once python is built, usually using pip is enough and works well

This way, !each of my repositories has its own nix file defining the list of dependencies and it can easily be built per system as well