Tried tons of ways, and finally found peace of mind with poetry:
It encourages to set up project specific definitions which are saved in the local pyproject.toml file. Keeping everything local and project specific, including the env definition, turns out to be a fantastic boon to reproducibility and sanity of mind.