"Interesting" thread from 2012 about replacing moreutils' parallel with GNU parallel in Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=597050 (TLDR: skip to message #75).
Mr. Hess makes a lot of good points, most I think I agree with, so I can sympathize with his frustration.
The point about transferring files seems interesting. I haven't used gnu parallel in remote/ssh mode aside from simple tests, but that seems wicked useful for some situations. Is there a real and simple alternative to running gnu parallel on multiple remote hosts? Is that something the moreutils version can also do?
Overall it has become my preferred Parallel implementation.
https://pypi.python.org/pypi/pssh/2.2
rdist I believe can operate in parallel as well.
Beyond this, devops tools e.g. ansible/puppet, etc. but they require a bit more or alot more setup, depending on the tool.
For most uses I find that ansible ad-hoc commands give a nice balance -- just a tiny bit of learning curve, and only a little bit different than typing the single SSH command that you want to run.