They can be installed at the system level, making the package available to any Python program on the system. This can get problematic if not all of your Python usage can function with the same versions of all system-installed packages.
They can be installed at the user level, sitting in your homedir instead of somewhere under /usr. This gives some isolation from the system defaults, but may just move the problem down a level - you still have the potential for version conflicts.
They can also be installed at the virtualenv level, and now the package install is specific to Python programs launched from that virtualenv. Upgrading a package can only break stuff in that virtualenv, it can't spread beyond it.
If you mostly use stuff from the standard library and stable packages that don't change their APIs much, you can go a long way without venvs before it actually bites you.