$ export PYTHONUSERBASE=/path/to/project-specific-python-user-base
    $ python3 -m pip install --user whatever

Everything now goes into that directory. Different projects on the same machine have different directories and so can't affect one another.