You can also

  pv /dev/zero | ssh your_server "cat > /dev/null"

Both yours and this one check upload speed.

To check download speed:

  ssh your_server 'cat /dev/zero' | pv > /dev/null

or based on your suggestion:

  ssh your_server yes | pv > /dev/null

And if you don't have a server to login to, you can find a big file on the web and use that to check download speed:

  curl -s http://some-place.com/big-file | pv > /dev/null

Though curl also reports speed, so I guess you may as well just:

  curl http://some-place.com/big-file > /dev/null