The server might have the wrong Python version available (2.x vs 3.x, multiple evolutions of 3.x with big features added in each point release). Et cetera, et cetera.
Or you could just do it with bash or a static Go binary and be done with it. Portable, works pretty much everywhere.
Imagine a situation where you'd need to perform an operation on a hundred servers, would you either transfer one static exacutable or build an virtualenv in every machine and download correct versions all of the related libraries?
That's the point of using virtual environments, so that you can run the Python version and libraries that you need. Also, as of 3.3, Python ships with venv which means you don't need to separately install virtualenv anymore. It's all very portable.
It really isn't as portable as one might wish.
Not all operating systems carry Python >3.3 by default, that would mean installing backports or unofficial repositories on production machines.
Or you can use a static binary that just works.