"Start moving large compressed files (gigabytes) and you'll find that out the bad way."
Not if you do that over SCP / SSL / TLS that said... (and the bottleneck is the network, not the symmetric encryption used once the public/private symmetric key exchange has been made).
So, yeah, everytime I copy big files I'm using scp (small files too actually but it's just out of convenience).