Hacker News Reader

Comment by tombert | Hacker News Reader

I’ll give this a go, though honestly for the stuff I use Parallel for, I don’t think that Parallel is the bottleneck; usually I use parallel to queue up like 50 ffmpeg or imagemagick jobs, and the vast vast vast majority of the time for those (I would think) would be consumed by the main program, not Parallel.

Maybe I have a really atypical workflow on this? Or maybe I’m misunderstanding something?

View on HN

I find parallel to have the best UI (beside the citation thing).

Whenever I use xargs (which is available more often on remote servers) I struggle to find the right invocation. And I learned about parallel way after using xargs, so I don't think it's just my habbit.

Similarly to you my use cases are relatively few (hundreds?) long-running commands, so the performance of the implementation is less important.

    touch ~/.parallel/runs-without-willing-to-cite
    echo --sshdelay 0.09\n--eta >> ~/.parallel/config

I used to use GNU parallel for automating some server management tasks (this was circa 2009). For long-running or (typical) non-interactive tasks, it's fine, but if you're waiting to see output from running something that should be really quick, the startup time is (or was) frustratingly long.

I ended up writing my own parallel execution code in shell, and it was much, much faster.

Is it the startup time, or the fact that parallel buffers and combines the outputs in spawn order?

If I don’t care about stdout, I usually use xargs’s parallel mode instead.

Reply on news.ycombinator.com