Hacker News Reader

Comment by Someone | Hacker News Reader

”But sometimes three is not enough; there is now a proposal circulating for a fourth kernel polling interface”

I’m not convinced. Can anybody explain what’s the set of desirable properties of polling interfaces, and why we need at least four different interfaces to implement all of them?

View on HN

> I’m not convinced. Can anybody explain what’s the set of desirable properties of polling interfaces, and why we need at least four different interfaces to implement all of them?

I'll give you a partial answer.

One desirable property is that each iteration of your event loop is not O(n) with the total number of descriptors being watched. select() and poll() are flawed for that reason—the entire list of file descriptors is passed in on each iteration and has to at least be compared to the previous iteration. No one writes things using these ancient interfaces anymore. (That's a bit unfortunate given that all the modern interfaces are single-platform, so everyone who cares about portability needs an abstraction layer, but it is what it is.) epoll() is better.

The kernel doesn't break old programs, so old interfaces stick around, basically no matter how bad they are. There are three interfaces now, so there have to be at least three. If there can be only three or if there have to be four comes down to if epoll is (or can become) good enough or not. If folks keep coming up with new requirements that can't be met with existing interfaces, there will just be more and more interfaces over time...

So, the answer is “we don’t think we need three, let alone four, but we happen to have two bad ones that we don’t want to get rid of”?

If those old ones don’t have any unique properties, couldn’t they be implemented on top of a single syscall, or is the syscall interface sacred on Linux, and calls cannot be retired?

The syscall interface is considered to be Linux's public API, and must very strictly never break backward compatibility. They cannot retire syscalls.

What tends to happen is that kernel is changed to implement the old syscall using the new infrastructure on the kernel side. There's no real cost to having another syscall number used.

But it grows the amount of code in the kernel, making it more likely to be buggy, more so if some of these old interfaces get used less and less (and, hence, likely tested less and less), and the underlying root interface gets refactored again and again to support new polling interfaces.

We are generally talking about very simple wrapper functions here that don't have a lot of scope for hidden bugs to creep in.

no way to instrument old source to compile old interface into new interface given it's proven equivalent ?

It's still reasonable to write things using the old select() or poll() interfaces, as long as it isn't something that has to scale up to thousands of file descriptors.

I can tell you some undesirable properties, which is what keeps motivating people to think of new ones.

Bad property 1. Limited number of things you can block on. select(2) can only do FD_SETSIZE descriptors.

Bad property 2. Entire list of descriptors needs to be re-examined on every blocking call. select(2) and poll(2) have this problem. Entering the syscall, the kernel needs to copy the list of descriptors, and after returning from it, the application needs to scan the list to see what changed - these are both O(N) operations. (Aside: Windows's WaitForMultipleObjects suffers from the former problem but not the latter, but has bad property #1 since it's limited to 64 handles.) epoll and kqueue fix this by having the kernel maintain the list of descriptors across calls, notifying you of which ones change.

Bad property 3. Too many syscalls, which epoll suffers from and this latest one is trying to solve. kqueue(2) tries to mitigate this by allowing you to batch add/remove descriptors, rather than doing a separate syscall for each.

Backward compatibility. Linux can never remove the older interfaces in this case. Only extension or creating new ones is possible. If you want new properties, you need a new interface.

And with the kernel showing up in things like the spacex rockets, that is the sensible solution.

Reply on news.ycombinator.com