I ended doing the following for those who are in the same situation:

    pip freeze > pip_list_with_sudo.txt
    sudo pip uninstall -r pip_list_with_sudo.txt

So now let's start doing things right with virtualenv...