Hacker News Reader

Comment by andrewstuart2 | Hacker News Reader

> 12. Write to stdout for useful information, stderr for warnings and errors.

I would constrain this even further to "Write to stdout only if information is useful as input to another program."

Even useful information for a human reader can make downstream integration overbearingly complex if the information is intermingled with a lot of extraneous, albeit human-readable, information. Machine-readable layouts (structured somehow: csv, tsv, json, xml, etc.) are vastly more useful for integration.

View on HN

>I would constrain this even further to "Write to stdout only if information is useful as input to another program."

May not always be possible to know this in advance.

Might be of interest:

https://www.google.com/search?q=the+unix+programming+environ...

http://jugad2.blogspot.in/2014/09/my-ibm-developerworks-arti...

Also, stdout and stderr can be redirected independently, and either one can be redirected to the other (cmd args 2> file, 2>&1, 1>file, 1>&2, etc.). There are even more advanced options. "man bash" has the details.

Edit: Fixed a URL typo.

> I would constrain this even further to "Write to stdout only if information is useful as input to another program."

No. `foo --help' should never ever write to STDERR. It breaks piping the message to `less', so I need to either juggle with file descriptors (bash) or use special syntax sugar for this juggling (zsh).

Whatever lands in `less', it's hard to be considered "input to another program", because it's mainly message for human that just happens to be paged.

Completely agreed. In addition, the following point

   > 8. Don't go for a long period without output to the user.

should be controlled by a switch and off by default. I hate chatty programs (only if there is an easy way to tell git to shut up).

on by default. If it's too noisy, you can look for how to make it shut up, and you will find it. If it's too quiet, you will look for why it's hanging, and not find it (because it's not hanging, just slow).

Shouldn't all programs be, besides their functional output, quiet by default? Isn't that why the "verbose" option exist?

Have you ever used wget? Do you think it's too chatty?

There is a signal, SIGINFO. For example, see BSD's `cp(1)'

     If cp receives a SIGINFO (see the status argument for stty(1)) sig-
     nal, the current input and output file and the percentage complete
     will be written to the standard output.

That is seriously neat. But! The user has to know this. Surely, if you're trying to copy a 6GB file, it's better to take a lesson from wget and print a progress indicator than force the user to (1) read the man page, and (2) keep polling the program to ask how it's doing.

"Write to stdout only if information is useful as input to another program."

Maybe devise a new std console channel for this integration purpose alone? "stdipc" (as in standard inter-process communication) maybe?

That's why they're called standard INput and standard OUTput.

"ipcin" and "ipcout" then?

I strongly disagree. A CLI is fundamentally for the benefit of humans interacting with the application, and not for the benefit of interprocess communication. When run interactively, a CLI should print human-useful information on the console.

Conversely, a structured way of exposing program outputs and state should be the preferred way of interprocess communication.

Because of convention and deliberate design choices, on the Unix command line, these two often find themselves in conflict. In my opinion, the solution isn't to compromise human usability to support machine-consumability of outputs.

Disagree.

A CLI is fundamentally for executing from the command line. Thus, embracing the power of the command shell is best practice.

If you want a human to see it, put it on stderr.

If you want a machine to see it, put it on stdout. Also I strongly agree with making it "structured" output. At minimum, a line-oriented record output is fairly easy to process downstream.

EDIT: If it is an interactive console application (REPL), then stdout is okay i guess.

Would a good compromise be to provide -q|--quiet to suppress extra, human readable messages when piping out to a different app?

yes, that would be a good compromise

Another option is looking at isatty. I'm not sure how I feel about either.

Please provide a flag that bypasses the isatty check. I use nohup for anything slow or hard to rerun and then tail and/or grep the nohup.out file.

Yeah, I think that's definitely good advice. isatty can be a good way of deciding which defaults to use, but it shouldn't be the only way of accessing functionality. Note that the flag might say "pretend isatty said yes/no", or there might be a few flags that independently mediate all of the things set based on isatty.

Personally, I'd just say use a tab-delimited table by default. And put columns that may contain whitespace last. This way it's human readable and more easily awk/cut parseable.

Unfortunately, with e.g. `docker ps`, I can't do as much as I'd like, because it's purely human readable by default. For example:

      ~  docker ps | awk '{print $7}'   
    PORTS
    Up
    ago
    Up

To suppress the human messages while piping stdin, I add `2> /dev/null`.

There are plenty of times that that's appropriate, but note that it also hides genuine error messages on failure. It's not a good solution for making output less chatty - that's precisely what a `-q` option is for.

It may be even better to be quiet by default and explicitly turn on the verbose mode when the "extra, human readable messages" are actually needed.

For one, STDOUT and STDERR both go to the console, so it's very feasible to do both. Additionally, there are quite a few machine-readable layouts that are very very human readable (table or tsv, for example, as a huge swath of existing CLI programs output by default).

I can very easily understand the output of `ps` or `ls` and I can very easily capture their output with a subshell: `for file in $(ls)` or `for process in $(ps -opid | tail -n +2);`, etc.

The more ways there are to accomplish the same goal, the greater the number of users who can figure out some way of automating what they want to do. So yeah, make output format an option, with `--ouptut` or `--format` for those who want to use `jq` or some other tool, but strongly prefer defaulting to table output so that standard tools (awk, cut, etc.) can be integrated easily.

Reply on news.ycombinator.com