Most projects don't need either.
I only use pipenv on a work project with a hundred dependencies, and venv on one other large side project. Everything else is installed with --user, especially dev tools like black and pyflakes. The user folder gets cleaned up every few years when you upgrade to a new version of python (rm ~/.local/lib/python3.5).
Has worked fine for a decade or two, no conflicts. I think some folks get spooked at python packaging and overcompensate in response, but it's pretty easy to troubleshoot when you've learned how it works.