The Self-Pipe Trick Explained | Hacker News Reader

The Self-Pipe Trick Explained

sitepoint.com

74 pointsdkarapetyan12 comments

This is a very good post because it explains the reasoning behind the code very well. It starts out with explaining the problem, and presents a simple solution to the problem. Then it explains the short-comings behind the solution and then refines the solution further. I find that it is exactly the way I learn, even my moderate understanding of somewhat complex stuff has always evolved from my understanding of simpler stuff. Wish more people wrote this way :)

If you're on Linux you can skip implementing it yourself and use signalfd(2) instead.

And if you are on a BSD/OSX, use kqueue(2). There is no reason to deal with the traditional signal interface the old way any more, just convert signals into event streams with file descriptor readiness like (almost) everything else.

(The almost is processes; Capsicum which is in FreeBSD and being ported to Linux lets you get a file descriptor for a process via pdfork(2) which you can poll too. Normally this doesnt matter as you can wait on the SIGCHLD anyway, but you can pass non child process file descriptors via unix sockets allowing waiting on other processes too).

How is appending to a global array re-entrant? If this was c-code this would be suspicious. Is there something in ruby that makes this safe? For example if the append code is:

    1. array[size] = element
    2. size = size + 1

then we could get:

    1. array[size] = element (first signal handler is run)
    2. array[size] = element (second signal handler is run)
    3. size = size + 1
    4. size = size + 1

and the array is corrupted

I am assuming that the global queue is written in a way that makes enqueuing and dequeueing an atomic operation that cannot cause a control switch. This code will ensure that re-entrance is unpermitted only during the relatively short time it takes to enqueue or dequeue an element as opposed to making the whole signal handler code non-reentrable.

I think it works because signal handling in ruby is implemented within its own queue. It looks like when signals happen in ruby they are added to a buffer and then threads check this buffer when they are running. So threads will only run signals when it is compatible with the global interpreter lock to do so. So as you said this would make the enqueing/dequeuing atomic because they are protected by the GIL.

On Win32 select() is called WaitForMultipleObjects() and it can wait on Events (similar to condition variables) as well as pipes, so such a trick is unnecessary. Not that the extra pipe is a serious performance loss, but it's heavier than needed for the task. But WaitForMultipleObjects has its own problems.

> But WaitForMultipleObjects has its own problems.

Could you elaborate on this? Even though, I tend to like Unix I thought that WaitForMultipleObjects was a better solution here.

I miss Jesse's work. He hasn't blogged or tweeted in a very long time, but always put out great stuff.

Me too. Jesse's books (and Eric Wong's mailing list posts) made approachable and opened my eyes to an exciting and fun world of programming outside of Web Development. Their books and writings have been a gateway drug to a very enriching learning experience.

This is also how ngx_pagespeed gets back onto nginx's event loop after finishing some work in a background thread.

This was brilliant! Very easy to follow for someone who is not very skilled in Unix.

Reply on news.ycombinator.com