HNHacker News
TopNewBestAskShowJobs

ole_tange

90 karma · joined August 14, 2013

submissionscomments
ole_tange··on [dead]
Ahh, but Sir, I think thou did not test it. Otherwise thou would have written differently.
ole_tange··on [dead]
The examples in the GIF-animation can be emulated by GNU Parallel using this:

To make the emulation easier, make a simple alias:

  alias par_emul="parallel -j0 --ssh 'sshpass ssh' --nonall --tag --linebuffer"
If you want to supply a password run:

  SSHPASS=`ssh-askpass`
or set the password directly:

  SSHPASS=P4$$w0rd!
If the above is set up you can then do:

  orgalorg -o frontend1 -o frontend2 -p -C uptime
  par_emul -S frontend1 -S frontend2 uptime

  orgalorg -o frontend1 -o frontend2 -p -C top -bid 1
  par_emul -S frontend1 -S frontend2 top -bid 1
                                                                                            
  orgalorg -o frontend1 -o frontend2 -p -er /tmp -n 'md5sum /tmp/bigfile' -S bigfile
  par_emul -S frontend1 -S frontend2 --basefile bigfile --workdir /tmp  md5sum /tmp/bigfile
There are a few differences:

* GNU Parallel normally uses 'ssh' and recommends using 'ssh-agent'.

* With 'sshpass' you only have to type the password once.

* GNU Parallel does not have progress indication for the file transfer.

* GNU Parallel uses /./ to indicate which part of the path should be copied to the remote side: foo/bar/./sub/dir/file will append sub/dir/file to --workdir on the remote side.

ole_tange··on [dead]
If it is more: define a function.

    myfunc() {
       echo stuff;
       # More stuff
    }
    export -f myfunc
    parallel -Shost{1..10} --tag --env myfunc --nonall  myfunc
ole_tange··on [dead]
Use --line-buffer to get line buffered output.
ole_tange··on Deep Learning with Spark and TensorFlow
Does --retries count as fault tolerance?
ole_tange··on Moreutils
Just for completeness https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=518696 Had parallel been packaged at that time (Mar 2009), it would have pre-empted the parallel in moreutils (which was added June 2009).
ole_tange··on Command-line tools can be faster than your Hadoop cluster
See comparison here: http://www.gnu.org/software/parallel/man.html#DIFFERENCES-BE...
ole_tange··on Hints for writing Unix tools
You view was heard in the design of GNU Parallel: It defaults to newline separation, escapes the argument, and is for most cases a drop-in replacement of xargs.

This does what you would expect:

  echo My brother\'s 12\" records.txt | parallel touch
ole_tange··on Ask HN: What's your favorite command-line tool for working with data?
histogram, https://github.com/ole-tange/tangetools/blob/master/histogra..., So you got this table and you are not really in the mood of firing up GNUplot/a spreadsheet/R but you would like a quick bar chart here in the terminal. cat data | histogram
ole_tange··on Ack is a grep-like tool, optimised for programmers
Be aware that xargs does not deal nicely with race conditions: (Parallel grep) http://www.gnu.org/software/parallel/man.html#differences_be...
ole_tange··on Ack is a grep-like tool, optimised for programmers
Consider running 'parallel --bibtex' once.
ole_tange··on GNU Parallel – The command line power tool
You have more than 1 space from uniq. 2 options:

  parallel --colsep ' +' echo {2} {3}
or:

  parallel --colsep ' ' echo {7} {8}
ole_tange··on GNU Parallel – The command line power tool
A lot of the users of GNU Parallel have felt exactly the same way.
ole_tange··on GNU Parallel – The command line power tool
Remember to show them 'parallel --bibtex'
← PreviousPage 2 of 2