ssh-copy-id is great, but the connection argument must be quoted, so if you need to use a different port you must use the form:

  ssh-copy-id 'user@host -p1337'

Took me far too long to figure this out :\