Hacker News Reader

> Let's say you want to read a specific page from a file.

pread() is an unusual case. I'd be happy if poll() simply indicated read() would succeed, and I'm happy doing an lseek() before every read().

However a pread() with O_NONBLOCK could return EAGAIN and poll() would then know what pages I'm interested in.

> what action, within posix, would you take to change it?

The general (POSIX-compatible) strategy (as I indicated) has been to fork a thread off to handle reading so that my application is notified when the data is in the fifo. This works for me when I'm reading (say) five files from a slow (or remote) storage, but it's not ideal since it causes a copy.

poll can't guarantee that the next read will succeed because first of all doesn't know how many pages you are interested in reading and any relevant page in the page cache might be evicted between the poll and the read.

Also having pread, which is designed for concurrency and scalability, carry state around for poll (an unbounded amount of state btw: you can have multiple threads issuing concurrent preads on the same fd) makes it completely pointless.

There are linux patches floating to add flags to preadv2 so that on a page not present it returns EAGAIN, so that you can optimistically do a sync read an fall back to a thread pool in the slow case. It still doesn't involve poll and friend.

poll() can't guarantee that the next read will succeed anyway, because something else might drain the socket or fifo (e.g. another thread). Programmers simply use O_NONBLOCK anyway and use poll() is wait until the process should try again.

I think this would be pretty straightforward if O_NONBLOCK was allowed to have an effect for regular files. The rules would be:

1. If a read for a range of bytes on a non-blocking file descriptor can be wholly or partly satisfied from the cache, put that data in the read buffer, and return the amount of it

2. If a read on a non-blocking file descriptor can't be satisfied from the cache, return -1, set errno to EAGAIN, and kick off an asynchronous task to load that range of bytes into the cache (an 'asynchronous load')

2a. If there is already an asynchronous load for that range running, don't start another one

3. When an asynchronous load finishes, mark that range as available

1a (surprise!). If that range of bytes was marked as available, remove the mark

4. If there are a nonzero number of available ranges for a file descriptor, return POLLIN from poll()

The way a program with lightweight threads (Node, Go, etc) would use this:

1. Open files as non-blocking

2. Make reads as usual

3. If a read succeeds, or fails other than with EAGAIN, pass the result to the thread and carry on

4. If a read fails with EAGAIN, suspend the thread, and remember the range it was reading

Meanwhile, in a background IO thread ...

5. Repeatedly poll all file descriptors for which there are remembered reads

6. If a file descriptor shows up as POLLIN, run through all the remembered reads for it, and make them again

7. If a read fails with EAGAIN, cool, leave it remembered and carry on

8. If a read succeeds, or fails other than with EAGAIN, pass the result to the thread and resume it

Would that work?

I haven't really thought about partial results of asynchronous reads. If a read asks for two blocks, and one gets loaded, should poll return POLLIN or not? I lean towards yes, because the program can make use of that data. That raises the question of whether to remove the availability mark after the following read. Again, i lean towards yes, because it means asynchronous reads have the same behaviour as ones which partially hit cache. The OS can carry on reading the rest of the range into the cache, but it won't signal the arrival of that data to the program. That should be fine, as the program will probably try to read it soon anyway.

If a program takes too long to get round to reading data once it is available, it might have been pushed out of the cache by then. Too bad. The read will fail with EAGAIN and the whole cycle repeats.

If two threads in a program independently read the same range, only one asynchronous load will be run, and only one thread will end up removing the mark. Programs will need to be careful not to trip up over this; they probably need to coalesce reads on their side of the API.

It sucks to have to retry every outstanding read for a file when it shows POLLIN, but i can't see a way to return information about which ranges are available through the existing calls. A program could keep its list of outstanding reads in time order, scan it from earliest to latest, and stop when it gets a non-EAGAIN result. If there is more than one available range, poll will return POLLIN again and the program will go back and pick it up. If asynchronous loads complete one at a time in order, this reduces the cost to O(1), with a smooth increase in cost as that is less and less the case.

Basically, what i'm saying is that everyone should just implement the kqueue API from BSD.

  2. If a read on a non-blocking file descriptor can't be
  satisfied from the cache, return -1, set errno to EAGAIN,
  and kick off an asynchronous task to load that range of
  bytes into the cache (an 'asynchronous load')

The way Windows and Unix operating systems are written, that "asynchronous load" will always be implemented by using a pool of kernel-scheduled threads. The filesystem is a stack of software, several layers deep, with tentacles reaching all over the place (e.g. modern file buffer and VM page caches are unified). Existing APIs and implementations simply do not support truly asynchronous operation throughout the entire stack. This is one reason why AIO on Linux is limited to direct I/O--to cut out all the middle layers--and even then it's not truly asynchronous (e.g. file opens). Windows asynchronous I/O just uses a kernel thread pool. So does FreeBSD. And so does every Linux proposal for buffered AIO.

If you want pollable file reads, just use eventfd and roll your own. Linux threads are lightweight enough that there's not much reason to care whether they're "kernel" or "user" threads. But pollable file I/O will really only be a win for streaming reads and writes, so you could just use sendfile(), in which case you're reading or writing to a pollable socket; you use a dedicated thread pool to pump data from sockets to the file descriptors, or vice-versa, without any copying.

BTW, Go doesn't need this solution (timeouts notwithstanding) because Go schedules multiple goroutines across multiple kernel-scheduled threads.

If you're really worried about file I/O latency, you should be at least as worried about VM pagefault latency. I disable swap on all my Linux servers. And I also disable overcommit. But my languages of choice (C and Lua) are capable of gracefully handling allocation failure (contrast Python or Go), and I implement my solutions to handle allocation failure without crashing the entire service. Too many other languages make the assumption that memory is infinite. (Alas, large parts of the Linux kernel do this as well, even with overcommit disabled :( Solaris and Windows are the only major OSs I'm familiar with that implement strict and rigorous memory accounting, permitting the system to stay alive (and consistent) under memory exhaustion. FreeBSD might as well, I'm not sure.

I agree completely. Pedantically, there is one other option: return an error like windows select does (WSAENOTSOCK). But then apps need special cases to deal with files and life is horrible.

> If poll where to return non-ready (and remember that poll has no idea which pages you are interested in), what action, within posix, would you take to change it?

readahead(2) ? posix_fadvise(2) ?

From man page of readahead(2), "readahead() blocks until the specified data has been read." The other one claims async functionality. But says it is not reliable.

An idea for how it could be implemented in linux: kernel would offer a new kind of file descriptor. It would have a limited, in-kernel buffer, much like a socket has an arrival space. Once you have it, call something like,

   newkind_page_copy(newkind_fd, disk_fd, page_count, timeout)

Timeout could be unlimited.

The buffer would behave like socket for the purpose of the read and error lists in select/poll/epoll. If you closed either descriptor, it goes to error-state. Within the kernel, it would watch real_file_fd, using code to similar to whatever is underneath inotify. Like inotify, this would not work for NFS.

> From man page of readahead(2), "readahead() blocks until the specified data has been read."

? What man page are you reading? The goal of readahead is to prime the cache and return immediately. http://man7.org/linux/man-pages/man2/readahead.2.html

I quoted from this one, https://linux.die.net/man/2/readahead

There is a second obstacle I have in mind, around the robustness of this. Imagine if we had an implementation of that which reliably returned quickly. But we take our time doing the subsequent read. By the time we do, the value has already been chased out of the cache. It is an edge-case, and you could probably avoid it in practice, but it would be significant effort to to verify your mechanism, and would create cognitive load.