So if in the future io-uring and friends become the standard can't that just be a library you could then use?
Similar to how in C you don't need the standard library to do threads or async.
So if in the future io-uring and friends become the standard can't that just be a library you could then use?
Similar to how in C you don't need the standard library to do threads or async.
Application developers of course want much higher level interfaces. They don't want to do a series of reads; they want "fetch_url". But if "fetch_url" is the lowest-level API available, good luck implementing an efficient streaming media server. (Sometimes we end up with things like HTTP Live Streaming, a horrendously inefficient protocol designed for ease of use in programming environments, client- and server-side, that effectively only offer the equivalent of "fetch_url".)
Plus, IOCP models tend to heavily rely on callbacks and closures. And as demonstrated in the article, low-level languages suck at providing ergonomic first-class functions, especially if they lack GC. (It's a stretch to say that Rust even supports first-class functions.) If I were writing an asynchronous library in Rust, I'd do it the same way I'd do it in C--a low-level core that is non-blocking and stateful. For example, you repeatedly invoke something like "url_fetch_event", which returns a series of events (method, header, body chunk) or EAGAIN/EWOULDBLOCK. (It may not even pull from a socket directly, but rely on application to write source data into an internal buffer.) Then you can wrap that low-level core in progressively higher-level APIs, including alternative APIs suited to different async event models, as well as fully blocking interfaces. And if a high-level API isn't to some application developer's liking, they can create their own API around the low-level core API. This also permits easier cross-language integration. You can easily use such a low-level core API for bindings to Python, Lua, or even Go, including plugging into whatever event systems they offer, without losing functional utility.
It's the same principle with OS and systems language interfaces--you provide mechanisms that can be built upon. But so many Rust developers come from high-level application environments, including scripting language environments, where this composition discipline is less common and less relevant.
While perhaps higher level libraries are written that way, I can’t think of a reason why the primitive components of IOCP require callbacks and closures. The “poll for io-readiness and then issue non-blocking IO” and “issue async IO and then poll for completion” models can be implemented in a reactor pattern in a similar manner. It is just a question of whether the system call happens before or after the reactor loop.
EDIT: Reading some of the other comments and thinking a bit, one annoying thing about IOCP is the cancelation model. With polling IO readiness, it is really easy to cancel IO and close a socket: just unregister from epoll and close it. With IOCP, you will have to cancel the in-flight operation and wait for the completion notification to come in before you can close a socket (if I understand correctly).
Anyways, I've been playing around with implementing some async socket APIs on top of IOCP for Windows in Rust [1]. Getting the basic stuff working is relatively easy. Figuring our a cancellation model is going to be a bit difficult. And ultimately I think it would be cool if the threads polling the completion ports could directly execute the wakers in such a way that the future could be polled inline, but getting all the lifetimes right is making my head hurt.
For comparison, Haskell went with the library approach but has the syntactic sugar of the equivalent of `and_then` built into the language. (I am talking about Monads and do-notation.)
It's a bit like iterating in Python: for-loops are a convenient syntactic sugar to something that can be provided by a library.
It is a PhD level research problem to know if monads and do notation would be able to work in Rust. The people who are most qualified to look into it (incidentally: a lot of the same crew was who was working on async) believe that it may literally be impossible.