Hacker News Reader

Hey, I wanted to chime in and say that I too am underwhelmed with the support for async in Python libs. Coming from a node.js background, it falls short because you constantly have to think about whether async code is calling sync code, or vice versa, and whether you need an executor, threadpool, etc. Just a mess that gets in the way of productivity IMO.

However, things have been getting better, and in case you haven't seen it, AIORedis[0] appears to be the de facto standard for async in Python (a little better with 1,724,389 downloads this month). It's popular enough that it's been merged with the official redis-py driver[1].

[0] https://aioredis.readthedocs.io/en/latest/

[1] https://github.com/redis/redis-py/releases/tag/v4.2.0rc1

> In order for async to be popular it needs to be the default way of writing python. Right now its an after thought for writing “high performance” python.

Might this have to do with the fact that the (lack of) performance of asyncio is exactly the issue here?

I remember reading a blog post by zzzeek (the SQLAlchemy author who's also participating in our discussion here) about precisely this issue a while ago. Not sure but it might have been this one:

https://techspot.zzzeek.org/2015/02/15/asynchronous-python-a...

Yes, though I do think the python async ecosystem is gradually moving from a "3rd class citizen" to a "2nd class citizen". There are sync+async libraries like HTTPX that are gaining support. I don't expect it to really ever be a "1st class citizen" (supported as well as sync in pretty much every library everywhere) because of backward compatibility and ultimately async is just harder to program. Sometimes developer time is more important than the performance wins of async.

> Sometimes developer time is more important than the performance wins of async.

Also, a fair fraction of applications don't see a meaningful benefit compared to other things the developers could be spending time on. Async is not only harder to write in many cases but can force you to deal with new problems which might be harder to manage (e.g. if you use non-trivial amounts of memory, bounding your peak usage can be a challenge).

I've used async a fair amount and it's worked really well in network-heavy uses (HTTPX is really great) but outside of those the savings have been a lot less than people thought, independent of the language used (I'm thinking of a colleague who spent a lot of time trying to beat classic Python with Go and ended up with about a 10% win after a month or so of work because the problem had enough CPU/memory contention to prevent greater savings — that's not a trivial win but it definitely wouldn't have been enough to justify changing languages on its own).

> unless 100% of the the libs you want support it you will get stuck on some sync library.

What do you mean by this? Do you mean like an actual deadlock, or just waiting on some kind of IO? Are you trying to use async for performance?