Do you have to figure this out? Sure, it's nice and "pure" if everything is configured through Nix but there is something to be said about being pragmatic. Personally, I just enabled nix-ld[0] and use uv to install and handle my Python versions and package dependencies. Much, much easier.
[0]: https://mynixos.com/nixpkgs/option/programs.nix-ld.enable