Comment by jlokier | Hacker News Reader

Comment by jlokier | Hacker News Reader

Parent Full thread

Linux has WaitForMultipleObjects already:

fds are equivalent to Windows HANDLEs, and WaitForMultipleObjects() is equivalent to poll().

But, Linux also has epoll() which scales better for non-trivial numbers of things, and Windows has IOCP. So WaitForMultipleObjects isn't particularly special.

Both Windows and Linux have things that don't work with these interfaces. Nonetheless, Linux has been trending towards "waiting on all sorts of things" by makng more and more things into fds that can be waited on with pol;/epoll. Examples: timerfd, signalfd, eventfd. It's quite a unixy approach.

In fact, Wine already uses eventfd to implement WaitForMultipleObjects. This kernel change is just an optimisation, to speed up Wine, and a workaround for some distros setting Wine's max fd limit too low for Windows apps.

Futexes used to support waiting on multiple futexes, using FUTEX_FD. That was arguably better than the new patch FUTEX_WAIT_MULTIPLE, because in old Linux you could wait for futexes and other fds at the same time - it did work with "all sorts of things".

But FUTEX_FD was removed after searches online found no code using it, and kernel devs didn't like keeping it. (To my mind, this was a suprising, unusual breakage of system call binary compatibility) The new FUTEX_WAIT_MULTIPLE allows programs like Wine to wait for multiple futexes faster than before, but it's more limited than the old FUTEX_FD because you can't mix them with other things.

epoll only works with file descriptors and is therefore not equivalent. The whole point is WaitForMultipleObjects works with many different types.

You are confused. File descriptors on linux represent "many different types." They are not just for "disk files". Please see signalfd, timerfd, eventfd, inotify, let alone sockets (which themselves represent things other than IP sockets). FD is essentially like handle. epoll therefore works with many different types.

While the Unix philosophy is that everything is a file - that is not the case with Linux. Futexs being a good example. There was an attempt to integrate them but it was done incorrectly with inherent race conditions and abandoned.

The VMS and NT philosophies of everything being an object are a bit more general and easier to follow in practice.

what are you on about? I just gave you the specific examples: timerfd, eventfd, signalfd, inotify... these are all epollable fds in Linux?

Futex is a special case because futexes themselves are quite special. There is no userspace equivalent to them in Windows anyway as has been mentioned. Windows Events are similar to what is provided by eventfd, but not as featureful.

Probably the core of the issue here is that VMS and derivatives tried to hard to fit everything into a generic handle/fd interface while UNIX historically came with a smaller core with many important API surfaces (like signals or timers) only relatively recently getting absorbed under the unified handle/fd interface, giving the impression it's an afterthought.

It's also not something that works on all Unices (e.g. afaik macos has no timerfd). But in all fairness, that problem only exists because OS speciation, as it's biological counterpart, is not as clear cut a concept as one would desire.

Macos timers can be waited on via kqueue FWIW.

I know; but they cannot be closed with close, or passed via unix domain sockets to other processes etc etc. I _think_ this whole thread is about "purity", whatever that means.

Futexes currently have _no_ kernel state besides in the threads that are currently waiting on them. There's no futex_create system call for instance. It's litreally just a call to "sleep until this memory address changes or a timeout occurs". There's not really anything to make a type around, which is why FUTEX_FD was doomed to failure and they rightly backed it out.

You may be surprised to learn that some windows handles are actually just addresses with no kernel state too. A great example is an HMODULE.

In fact windows actually reserves a block of user address space that will never be allocated to disambiguate memory vs non memory handles.

Its really a pitty these two great systems refuse to learn from each other.

What does that have to do with anything? You wouldn't use WFMO to implement a futex in Windows. And regardless of anything in Linux, defending WFMO seems like a strange hill to die on. It's not a great API.

I'm calling your bluff too. Which one of the supported waitable handles in Windows are just addresses with no other state? I'm more surprised because they all need an access mask at a minimum, and I thought they all involve an ObCreateObject, even a Mutant... these dusty corners of the kernel are visible through the DDK. Anyway, I would be glad to be shown wrong.

I’m not sure why theres such animosity in your replies. The goal here is not to defend windows but to point out that handles need not aways refer to kernel objects.

It would be possible to provide an orthogonal handle based API even if windows doesn’t always live up to that ideal.

Ultimately it seems these systems are just too big to maintain a cohesive design - but it is still an ideal to aspire to.

The handle is a _kernel_ address... backed by a validly constructed kernel object.

EDIT: And since you dirty edited, can you point to an example of HMODULE or any other dataless HANDLE being used on the syscall interface?

Handles are not kernel addresses in NT and windows will not intentionally leak those to user space. Handle accesses are always through a layer of indirection in the object manager.

You will find that windows never makes guarantees of what handles actually are - in case it wants to change them.

In windows NT most GDI handles are user mode and not kernel objects. Other objects may or may not be kernel based depending on the version and whims of the implementor.

I mean, they're very clearly offsets into the HANDLE table kernel space.

And can you WFMO on user space only HANDLEs?

Ah you edited your response? An HMODULE.. That is weak dude. Calling and HMODULE a HANDLE is cheating... by that definition a UINT is a HANDLE since you can cast it.

Can you use HMODULEs orthogonally to common APIs that use Handles? That's what really matters. Just because something is 4 or 8 bytes wide and you call it the same thing isn't interesting. Like can you pass HMODULE to WaitForSingleObject? Oh Ok, I guess that's mean because to be fair, what does "waiting" on a DLL mean.

Ok, well surely you can pass that HMODULE to CloseHandle, at least closing should be orthogonal.. Why don't you try that? I'll wait.

So what point are you trying to make here? It sound like you're just jerking everyone around, tbh.

Edit: Moreover, it's a bit ridiculous to say that an HMODULE is just an address with no kernel state. It uniquely identifies the loaded DLL, so it a key to a tremendous amount of kernel bookkeeping about the loaded module.

Windows has WaitOnAddress() which is similar to a futex, and likewise isn't a HANDLE.

"File descriptor" is just a weird spelling of "handle", for historical reasons.

The vast majority of the interesting types _are_ files under Linux. You can use epoll with files, signals, timers, sockets, pipes, kernel semaphores, other epoll descriptors, page fault handling descriptors, etc.

FDs are pretty much HANDLEs of the nix world.

HANDLEs are better. And on Windows you get HANDLEs to processes. We could have that in *nix land, you know.. something like opening /proc/$victim/status and using it to wait for the process and keep its PID from being reused until this is closed.

> We could have that in *nix land, you know.. something like opening /proc/$victim/status and using it to wait for the process and keep its PID from being reused until this is closed.

We are going to soon: "The 5.3 kernel also adds the ability to pass a pidfd to poll(), which will provide a notification when the process represented by that pidfd exits." (https://lwn.net/SubscriberLink/794707/905eb6b5b7287e77/)

Last I heard, epoll() was, ironically, not useful with actual file descriptors, because file I/O isn't blocking on Linux [1]. I'm not sure if this problem has been remedied in recent years.

[1] https://stackoverflow.com/questions/8057892/epoll-on-regular...

Reply on news.ycombinator.com