Comment by Bogdanp | Hacker News Reader

Comment by Bogdanp | Hacker News Reader

Parent Full thread

Here are my reasons:

* I am not a fan of asyncio's API (`ensure_future`), docs (though I understand these two are improving in 3.8) and of the fact that it is incompatible[1] with much of the existing ecosystem of libraries (eg. the SQLAlchemy ORM or any existing libraries that talk to... anything over the network (postgres, redis, memcached, etc.) which need to be written from the ground up in asyncio style or rewritten so their protocol parsing logic is completely separate from the IO that they do (not a bad thing, mind you, but taking battle-tested libraries and rewriting them from the ground up is not the greatest idea)), tools and even builtins (queue.Queue vs asyncio.Queue).

* I don't buy into the claimed benefits of explicitly marking yield points: any sufficiently complex code will have enough yield points (`async with`, `async for`, `await`) that reasoning about concurrency in the program isn't going to be any easier than if it had implicit yield points.

* I think ergonomics matter for developer happiness and programming with asyncio is not ergonomic in the slightest.

* File IO is blocking.

Put all of those together and you end up with a system where you can't use any of the popular, battle-tested, libraries out there, where your code is more verbose for minimal gain and where seemingly benign things (reading a file, using a `for` instead of an `async for`) may greatly impact the performance of your program all for a small increase in throughput (assuming it does increase!).

[1]: http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y...

My only counterpoint to this is two fold.

I know writing asyncio code is newer and therefore less established and that will certainly crop more bugs for less experienced programmers I’ve witnesses this myself.

With that said my 2 points are as follows:

1. Asyncio has comapbility for running threadsade code (asyncio.Lock, loop.run_thread_safe etc) which can take those blocking functions and run them without having to give up all the benefits of awaitable code. In my albeit somewhat brief testing SqlAlqchemy worked okay in this context (I know peewee does its my personal ORM of choice)

I happen to agree however that fileIO should’ve been the first thing they implemented as async as it’s the biggest gain instead of leaving it up to the community to re implemented

In fact thinking about it it’s the explicit use of Futures that allow me to often bridge sync code like an ORM into asyncio without too much trouble IMO

2. Given that, did you test any of your io heavy code against an asyncio version? I’m just curious. I’ve been seeing personally huge bumps in performance by switching our production apps from Django to Tornado

> I don't buy into the claimed benefits of explicitly marking yield points: any sufficiently complex code will have enough yield points (`async with`, `async for`, `await`) that reasoning about concurrency in the program isn't going to be any easier than if it had implicit yield points.

agree. Another point where programmers seem to not be realizing this and I can only assume they need to gain more experience to see this.

1) This is fair.

2) Explicit yield points certainly save you from most of the classic threading errors, and having to use special maps and datastructures everywhere.

3) Other than tying into (1) I guess this is a matter of taste.

4) Supposedly this refers to the `open` function? This can be solved using the `aiofiles` module, but again I suppose this ties into (1).

> Explicit yield points certainly save you from most of the classic threading errors

You know what also saves you from most classic threading errors? The GIL, and yet people writing multi-threaded code in Python still use locks and thread-safe datastructures, because most errors is not enough.

Explicit yield points when the developer is forced to yield does not bring much more safety than executing normal Python bytecode.

Yes, a lot of it boils down to the ecosystem split for me.

> 2) Explicit yield points certainly save you from most of the classic threading errors

Can you provide an example of such a threading error? At most I've heard people claim that asyncio removes the need for mutexes[1]. I think that may be true at the local level, but as soon as your function needs to synchronize with other async functions in the system that ceases to be the case. At any rate, that's not enough of a win to justify the split in my opinion.

> having to use special maps and datastructures everywhere.

The GIL already protects you from corrupting builtin datastructures. Neither the GIL nor asyncio protect you from corrupting your data (the need for some form of synchronization), however.

> 4) Supposedly this refers to the `open` function? This can be solved using the `aiofiles` module, but again I suppose this ties into (1).

And most operations on opened files (like `read()`) as well as any OS bindings provided by the `os` module.

[1]: Clearly, asyncio itself still thinks they're useful: https://docs.python.org/3/library/asyncio-sync.html#asyncio....

Wrt (2): Personally I find coroutine programming to be unintuitive and often lead to unreadable code, where the control flow is hard to reason about. Additionally, it introduces the honestly idiotic programming task of profiling to find chunks of code that "don't yield enough" and then breaking them up to yield more. The possibility is always there that some engineer on your team will commit some nasty blocking API handler that stalls your endpoint.

Threads don't have any of these problems. Conceptually they are sequential code. You just have to clearly state what the communication model is between threads, and use a good one. The actor model works really well.

> 2) Explicit yield points certainly save you from most of the classic threading errors, and having to use special maps and datastructures everywhere.

"classic threading errors" don't apply to the users of a web framework where you are programming logic that shares nothing with other threads. Additionally, the most free-of-race-conditions monolithic application loses all of that advantage the moment it is placed in a multi-process environment, where other copies of it on the same host or other hosts are competing for the same resources. There is no non-trivial web application today that is deployed to production in a single process on a single host only. Web applications are not scaled vertically.

> Web applications are not scaled vertically.

Should be rare, but believe stackoverflow was doing that for a while.

Reply on news.ycombinator.com