Asynchronously Opening and Closing Files in Asyncio
nullprogram.com
nullprogram.com
I would also like to point out that asyncio made a huge mistake here. IO is not limited to interactions with the network buffer. I don't know how something called asynio can be released in to stdlib, and not include disk io.
The fact that there is no actual async file io on the OS doesn't matter. Dealing with files is a core concept in almost every application, and so pushing this to stdlib and hoping the python ecosystem takes up the slack, or people handbake threaded access in themselves, is a joke.
Some of this is a little dated, but a friend of mine wrote this and included my complaints about file io in there somewhere too:
https://web.archive.org/web/20171206105117/https://veriny.tf...
Some problems might require extensive file IO, sure. But most don’t, and something like reading from a database or making a HTTP request is orders of magnitude more common.
From personal experience, even for serving files, I find myself relying a lot more on web services rather than a local disk.
> Then I found curio. And I realised how badly designed asyncio was.
That quotation, the rest of the text at that link, and the comment I'm replying to, all hinge on the fact that asyncio was the first async library available. There are two consequences of this:
1. asyncio was used to drive library design forward. Yes it's easy to look at curio (or trio) and say that the design is obviously better than asyncio, but even the author admits they only realised it after they saw curio. This shows how legitimately difficult it is to design an async API. The best way to come up with any design is really to start off with a first attempt and use that (especially by analysing its weaknesses) to come up with a new one. That's what happened here; I don't think curio could ever have been created without asyncio coming first to inspire them, even if it was partly the mistakes that inspired them. Indeed the asyncio API has changed a lot since that post was written, partly learning from its own mistakes directly (e.g. there's almost no reference to the event loop in the API now) and partly learning from the API in curio/trio (e.g. asyncio.run()).
2. asyncio was used to drive async language features forward, many of which are relied upon by curio and trio for the designs referred to in that post. These were ultimately used in asyncio too (as you said, many arguments in that link are now out of date - the main objection is surely "Ignoring the generator API" which is no longer true). I don't think an external library could have pushed language evolution as quickly as a part of the standard library did. For language stability reasons, the standardised version of asyncio always lagged the language features by one Python version.
In many ways, the primary job of asyncio was to get something working to allow these two things to be iteratively improved. Missing out file IO is reasonable because it still allows the library to accomplish those goals, and people can still call out to a thread to do it (which needs to have a good API anyway because there's no avoiding it for CPU-bound tasks). In many platforms, that's all that would happen under the hood anyway.
------
A couple of specific responses to the objections in the link (many are simply "that's no longer true" and I'm not going to try and list all those):
> Additionally, writing is not async. Is all your data sent when you write? Nope! ... The work around, of course, is to call writer.drain(). Why it doesn't do this automatically, who knows?
Sometimes you want to queue up data to be written to a socket and then do something else (in that coroutine), and sometimes to you want to queue up that data and wait until it actually gets written. The only time I've used asyncio in a real-world project, I've wanted the former; waiting would've been a disaster! The obvious way to allow this flexibility is to split up those two operations - and they really are separate operations - into two separate functions. This isn't a "workaround", it's clean design.
The alternative is to make it all in one function. That still technically allows both use cases: if you want to just queue up data then you can spawn a new coroutine for doing that with a queue/stream to send it data from other coroutines. But that's causing one of the two API uses to have way more code than the other, whereas having to choose between calling one or two functions is really not a big deal. It also means there's the runtime overhead of two queues, since the internal queue of data to write still really exists, and it's harder to arrange for smaller packets to be coallesced for writing... and so on.
> Even the call to enter thread-land is bad: loop.run_in_executor(executor, callable). None is almost always passed as an argument to executor.
You almost always don't want to pass None as the executor - I suspect the argument is deliberately first to discourage that! It's pretty normal in a threaded application to have different threads (or thread pools) for different types of work. This API lets you use different executors representing different types of thread or thread pool. Even if you only need one "processing" thread (/pool) for now, it's still good hygeine to be explicit about it.
> Once you're in thread-land, you're basically SOL if you want to interact with async-land.
Now I do concede this one could be a bit simpler. If you async thread is ultimately in charge of everything then the pattern that worked for me was a queue to pass the data to a processing coroutine, which in turn passes the data to the thread:
async def process(self):
while True:
input_data = await self.queue.get()
if isinstance(my_data, Exception):
raise input_data
else:
output_data = await self.loop.run_in_executor(
self.my_executor, processing_fn, input_data
)
yield output_data
(To be clear: my_executor is shared between multiple instances of the class with this method, so multiple connections end up sharing the same thread pool for this type of work, but there are other executors for unrelated work.) You can then use an `async for` to loop over the processed data.If you want something more complex then you can use asyncio.run_coroutine_threadsafe. This was introduced in Python 3.5.1, which is probably after that article was written (so that's a case of "no longer true").
This definitely doesn't sound right. Running strace, you need to specify -f in order to trace threads, otherwise only the main thread is traced. gdb does stop all threads on interrupts, but only by default for convenience, and I don't think it's atomic.
Because of this, LD_PRELOAD is not the easiest way to delay system calls, strace -e inject:delay is.
$ cat > a.c << EOF
#include <stdio.h>
#include <sys/types.h>
#include <fcntl.h>
#include <pthread.h>
#include <unistd.h>
void *start_thread(void *arg) {
while (1) {
puts("thread");
sleep(1);
}
}
int main() {
pthread_t thr;
pthread_create(&thr, NULL, start_thread, NULL);
while (1) {
puts("main");
close(open("/dev/null", O_RDONLY));
sleep(1);
}
}
EOF
$ gcc a.c
$ strace -o /dev/null -P /dev/null -e inject=openat:delay_enter=3s ./a.out
main
thread
thread
thread
thread
thread
main
thread
thread
thread
thread
thread
[ and so on. ]Often times the price you pay to send the job to the queue, yield for another coroutine, wait for IO task to finish, receive a notification back from the thread pool and resume is higher than doing the blocking call. Under high load it might have some small benefits but my hunch is that it's unlikely to buy you much.
Luckily, io_uring officially made it to the linux kernel few months ago and can solve these problems in a much more efficient manner. *BSD solved this problem long time ago, in 2002, when kqueue was introduced.
I have used Asyncio for an API I built for https://www.abstractapi.com/ in order to be able to manage high load and avoid being locked by network latency.
It has been painful and I almost thought about migrating to Amazon Lambda to distribute the load but it feels like a bad use case for Lambda when you're actually only want to perform async stuff and not lock a full thread.
SCan't believe we still don't have a proper async lib for Python in 2020. Still have to test Curio but the whole thing makes me seriously reconsider which language I'll use for my next project. Async is something I need to handle more and more.
Ordinary Linux file i/o for example cannot be made non-blocking and ignores O_NONBLOCK flag. See open(2).
well reading and writing can, opening can't. there was also a good conversation about that on HN: https://news.ycombinator.com/item?id=17654550
https://lwn.net/Articles/810414/
For cases like that discussion about opening lots of files over NFS this should allow it to be done completely async. The actual time to do it won't be better on a per file basis but you'd be able to parallelize them and do other work while waiting on the opens to complete.
From the libuv docs [0]:
Unlike network I/O, there are no platform-specific file I/O primitives libuv could rely on,
so the current approach is to run blocking file I/O operations in a thread pool.
For a thorough explanation of the cross-platform file I/O landscape, checkout this post [1].
[0] http://docs.libuv.org/en/v1.x/design.html#file-i-o[1] https://blog.libtorrent.org/2012/10/asynchronous-disk-io/
Lookup the libuv threadpool for more details.
In my case it wasn't important to open log asynchronously, but it was critical to have async writes.
It turns out, depths of Python stdlib hide logging.handlers.QueueHandler and logging.handlers.QueueListener which allow to split log message handler into separate thread from log message emitters, making them communicate via synchronized queue.
Final solution is quite compact: https://github.com/Snawoot/postfix-mta-sts-resolver/blob/589...
Usage example: https://github.com/Snawoot/postfix-mta-sts-resolver/blob/589...
It is convenient that you can still use logging as usual, without any async obligations. You just have logging function which works in a fire-and-forget fashion.
> Asynchronous reads and writes would require all new APIs with different coloring. You’d need an aprint() to complement print(), and so on, each returning an awaitable to be awaited.
> This is one of the unfortunate downsides of async/await. I strongly prefer conventional, preemptive concurrency, but we don’t always have that luxury.
I was sceptical about Golang for a long time, but recently I've started practicing it. It's designed with concurrency in mind from very beginning and goroutines have advantages of both cooperative and preemptive multitasking. And also it doesn't have GIL, so your goroutines will use all available CPUs without any additional effort from your side.
From programmers point of view, coding with goroutines looks exactly like coding with use of pthreads (even simpler, in fact). So, there are no function "coloring", no need in sync/async library flavors and so on. But these "threads" also have cost close to cost of python coroutines. It sounds too good to be true, but I've rewritten bunch of my Python/asyncio code in Go and I can say it's true. For this reason (and many others I discovered later) I quit using Python and asyncio for network applications in favor of Go.