The thing is, your program is only as asynchronous as the lowest I/O level. It doesn't really make much sense to have your business logic take advantage of this, if your program will block on lower levels.
So you DB connection, caching, HTTP requests, disk I/O, etc must be asynchronous to really reap the benefits. Since Python has been synchronous for something like 20 years, there's a lot of historical code that simply won't play nicely with the asynchronous approach.
Hence the brand-new libraries.
it's called non-blocking sockets, precisely the thing that 'gevent' brings on the table without breaking conventions of other existing interfaces, contrary to what 'asyncio' does - https://stackoverflow.com/questions/42448664/async-generator...
And no, it's not only the lowest I/O level, since there are groups, queues, chains, and other primitives [1] that work on higher levels without special keywords too.
SQLalchemy last version is asyncio compatible now: https://docs.sqlalchemy.org/en/14/orm/extensions/asyncio.htm...
> network libraries and entire web frameworks for the sake of two new keywords
The trend in the community now is SansIO: https://sans-io.readthedocs.io
You create entire libs without the IO parts, dealing only with the business logic.
The you let the I/O libs to deal with the rest. This allows modern project to provides a sync and async API with the same protocol implementation. E.G: https://github.com/encode/httpx
Also, I'm pretty glad gevent has not been adopted:
- it's very hard to know if you are doing something wrong with gevent. Or doing something with gevent at all.
- gevent magic patching broke stuff in unpredictable ways, and the solution was to disable part of it: https://stackoverflow.com/questions/8678307/gevent-monkeypat...
- gevent didn't play well with stdlib threads and multiprocessing, (had its own thread pool, subprocess wrapper, and no multiprocess story) while asyncio has a very good interoperability story
- the asyncio pluggable loop meant we didn't need a perfect solution out of the box. Thanks to that design, we have been able to integrate the nodejs loop, like with the uvloop project, giving birth to stuff like asgi, fastapi and uvicorn. Asyncio can now comparable async perfs as Go. (gevent does that, but forces you to, and having libuv in the stdlib is not something the core devs would want).
- gevent is a 5mo package. It's 1/6 of the ENTIRE python installer.
- the coroutine based design means we have a standard api for async, no matter the backend or implementation. Which means solution like trio were able to emerge, and eventually the core dev are now considering including trio-like scoping in the next asyncio iteration.
There are plenty of problems with asyncio though:
- create_task is basically a goto. See trio for a solution.
- coro() does not run the coro task. In JS it does, and returns a future you can await, or not. This makes it very confusing to beginner that get lost in the await/return/create_task dance.
- there is no concurrency limiting primitive, cancellation is hard and back pressure mitigation is non existent.
I was very skeptical about asyncio, but after 3.7, it became quite nice. asyncio.run() made all the difference, ergonomically, and when dealing with errors.
it's been compatible for 5+ years already, Core API was driver-endependent even in 2015.
> Also, I'm pretty glad gevent has not been adopted:
When I say 'adopted' I mean integrated into the language runtime so that the entire stdlib is aware of it. Some of your points consider the state of gevent 10 years ago, that's not a benchmark for a conclusion on how successful a potential integration into the core language might be, and none of them point at inherent issues with the implementation that make it unusable compared to asyncio.
That's fair.
I'm also biased by the fact I really like to know when my code is suddenly going into async territory.
I like the JS model for this: it's explicit enough that there is little surprise (promises are now a standard, so if something returns a promise, it's async), but implicit enough that I don't have to care (no loop to setup, no coroutine instantiation between calling and scheduling). And it aged well: future can now just be awaited, and you can simply keep the callback api in non async function.
gevent is too implicit to my taste I guess. Now you could argue that we could make gevent calls explicit, but then it would be not that much different from asyncio.
I would really be happy if the community would settle in something like:
@autorun
async def coro(): ...
Then: coro() # this calls create_task(), and binds it to the upper nursery
I think it would strike the right balance from a usability point of vue.As for the interoperability issue, it's a pretty though nuts to crack.
The structured API for concurrency that Trio provides, in principle, doesn't depend on the said keyword-based approach - it's just what the author chose to use for the implementation. It's definitely not something that asyncio enabled for its users that wasn't possible without it. Here's another structured API for concurrency that doesn't depend on keywords [1]. It can be implemented with gevent.
[1] https://hackage.haskell.org/package/async-2.2.3/docs/Control...
I get that scoping trough nurseries is the trio selling point, but you do need to have the ability to run something "before" and "after" a lone async call, and I don't want to have to write a "with" block to wrap any single thing that I wish to await.
By having "await" as a universal construct, we allow trio (or anything really) to offer an additional level of granularity - scoping a group of async calls - without losing the smaller one: scoping a single call.
Oh no, that means I'll need 2 more floppies to distribute my software!
Not to mention python runs on constraints systems such as MicroPython, which features coroutines.
HN, the place were people complain about how electron is bloated then ask you do copy that.
Libraries will still have to be re-written and public interfaces will change. Might as well do it right, and not abandon the concept of functions having meaningful return values.
You build on callbacks to create futures and queues. That’s how you do async programming in any framework. AsyncIO works on top of this and so does Gevent. The only real difference between the two is that asyncio explicitly types functions that do IO. The fundamental advantage is little more than a typing decorator.
The reason that JavaScript was such a crapshoot of callbacks was because the majority of JavaScript developers had no idea what a future/promise was.
"I want this to happen in the background without blocking" should never have turned into this terrible dance of rewriting everything using new magic keywords so that you can then wedge it into an asyncio event loop. It's almost like developers have no sense of UX elegance.
> "I want this to happen in the background without blocking"
That is not the problem the asyncio is solving.
It is, actually! All asynchronous code amounts to having some things occur in the background while other things occur. asyncio just does it with a clumsy interface that breaks everything that already exists.
Your IO is (that's why the library is called "async io")!
The entire reason the library exists is because all IO blocks at the lowest level, even so-called non-blocking IO, because computer science abstractions don't change physics, which is why sockets require buffers. At some point in the chain your process is asking the OS to do some IO, and at that point the process gets a choice between twiddling its thumbs or doing something else and coming back to check on the result later.
The mechanical difference between using coroutines vs threads for IO, under the hood, is the process polling sockets itself vs letting the OS do it. The OS allocating chunks of time to checking different threads that are spinning their wheels waiting for an IO response that may or may not have arrived yet on an OS socket buffer is fundamentally equivalent to the process allocating chunks of time to checking different coroutines that are spinning their wheels waiting for an IO response that may or may not have arrived yet on an OS socket buffer. The only difference is which layer does the spinning, the OS's schedule timer or the process's event loop timer (which of course itself runs at the whim of the OS's schedule timer). Are there performance differences between those approaches? Yes because creating OS threads has more overhead because they're fundamentally more powerful, but that's a technical detail that should be hidden, not exposed. Introducing an interface that requires new magic keywords and behavior in the underlying requests so that everything needs to be rewritten is terrible when the actual desire is _always_ some grammatical variation of "I don't want every IO request to prevent using the CPU until it completes".