Hacker News Reader

Hacker News

Comment by boshomi | Hacker News Reader

Parent Full thread

boshomi

the git commit: pipe: use exclusive waits when reading or writing[1]

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

View on HN

I am always amazed at the quality of linux kernel commit messages.

Indeed. Those two lines in the test-case are particularly excellent:

  /* 64 processes */
  fork(); fork(); fork(); fork(); fork(); fork();

I can totally see most people, myself included, writing a dozen of lines instead, with a for loop, checking for parent/child/error return code, etc. Which might be preferable for some code somewhere deep inside the app logic that will potentially run on a resource-constrained machine. But for the explanation code to be read by a human, this one-liner is such a good choice!

Indeed, my personal rule of thumb is if the ratio of commit message to diff is greater than one then it’s a good commit. Larger the ratio better the commit and you would do better to read the commit (as opposed to skimming through them).

Chances are, you’ll learn something new.

From the linked text:

> In particular, the GNU make jobserver code ends up using a pipe as a way to limit parallelism, where each job consumes a token by reading a byte from the jobserver pipe, and releases the token by writing a byte back to the pipe.

Umm, that's what semaphores are for.

GNU make runs on a lot of different systems. There is not a semaphore API shared between them, but there will (almost certainly) be a usable pipe() on all of them.

But… if I could, I would probably use POSIX semaphores, no wait SYS-V, no, hang on, let me read some web pages… um… let me check my repositories and see which I've used before… both… hmm…

Seriously, I can't see getting so frustrated choosing that I'd just go with pipe(), but there is room for improvement in portable semaphore APIs.

I tried quickly to find the list of supported systems, but the main documentation pages don't mention them.

That said, pipe() is not "portable" either, so whether to use platform-dependent code for pipes or semaphores... no big difference.

> Seriously, I can't see getting so frustrated

If reading documentation and evaluating options frustrates you, you shouldn't be dealing with software development.

> That said, pipe() is not "portable" either, so whether to use platform-dependent code for pipes or semaphores... no big difference.

pipe() is in POSIX base since forever. If it is not portable, I don't know what is. Oh well C is more portable than POSIX, ok; but C does not do IPC at all (actually not even processes) so that's irrelevant. SysV semaphores are in XSI, POSIX semaphore are only in base since recently, previously they were in a Semaphore option.

So yes, pipe() is absolutely more portable. Works on all BSDs, GNU/Linux, embedded Linuxes, Solaris, dead Unixes, Android, QNX, Cygwin (where on the other hand semget seems buggy: https://cygwin.com/ml/cygwin/2019-08/msg00099.html), Mac Os, etc.

Except that now, thanks to the GNU make change in 2017, there are two families of jobserver where previously there was one. There is a family that sets the pipe file descriptors to non-blocking mode, and there is a family that sets the pipe file descriptors to blocking mode.

Alas, the twain do not mix. So anything that wants to interoperate has to know what family of jobserver it is interoperating with. (It has been well known in the Unix world for years that if all programs do not agree on whether a shared file descriptor is in non-blocking mode, difficulties ensue. Yes, nonblock_read() and nonblock_write() would have provided a way to solve this whilst retaining interoperability.)

* https://jdebp.eu/FGA/dont-set-shared-file-descriptors-to-non...

> Oh well C is more portable than POSIX, ok; but C does not do IPC at all (actually not even processes) so that's irrelevant.

C11 adds threads and mutexes

Reply on news.ycombinator.com