Hacker News Reader

Comment by jacob019 | Hacker News Reader

This is why I love gevent. Synchronous code becomes asynchronous automatically and it's rarely necessary to reason about the asynchronicity.

View on HN

I'll add something I've observed to that:

In asyncio and similar systems, the choice between "is this code syncronous or asyncronous" is made at declaration time - either you define a sync function or an async one, and you can only use it in that way.

In gevent and similar systems, the same choice is made at call time - there's no distinction between sync and async functions, instead you can call syncronously with "foo()" or asyncronously with "future = gevent.spawn(foo)" (I'm taking some liberties by calling the returned greenlet object a future but it can be used as such).

Exactly. gevent has been a joy to work with since day one. asyncio (even with the new syntax additions) has been a complete pain for me since day one. I still use gevent for everything - even in brand new Python 3.7 projects - and I see no reason to stop. Great API, great performance, "just works".

As much as I love gevent I can't claim in good conscience that it always "just works" for me. It tends to just work until it doesn't, and then you're stuck investigating how a dependency of a dependency just switched to using a network library implemented in C that therefore isn't monkey-patchable and causes process-wide stalls.

I've gotten good at diagnosing these issues over many years as a heavy gevent user, and it doesn't stop me highly reccomending gevent to anyone who will listen, but it's a caveat that should be mentioned for anyone new starting out.

What libraries have you run into trouble with?

grpc is one that we're struggling with at the moment, psycopg2 is another that won't Just Work (you need to use a special "adapter" that teaches the underlying C library to hook into the event loop). In general anything that is a) not implemented in pure python, and b) doing calls that you expect to be monkey patched (mainly network calls).

This is thankfully fairly rare, but it is something you need to be aware of.

There are pure python alternative implementations of these protocols and of mysql, anything pure python works well, but of course there are modest CPU costs. Then there are workarounds if you must use C libraries with blocking IO, like using a pool of sub-processes. But using C libraries for IO is pretty low level stuff, and making them concurrent is beyond the scope of python.

It definitely causes problems when dealing with native modules that have networking functionality, but I've almost never had a need to use those modules.

Also, wouldn't asyncio probably have similar issues with native modules as well?

That sounds like goroutine, you can call any regular sync function with `go fn()`, and a goroutine is started.

Yes it's the same thing - they are green threads, hence gevent. Go has the additional feature that those threads can consume CPU at the same time.

Seconded - I recently wanted to try some new async frameworks and libs (starlette, asyncpg, etc) but I'm so turned off by the async creep I decided to roll the same gunicorn/flask/gevent/psycopg2 thing I've been doing for a long time -- it's a very nice system really.

The extensive gevent monkey patching is a major turn off for me. Has it ever broken for you?

Never, it's fantastic. The monkey patching just adds context switching in all the right places, like IO and sleep. I've been using it for the better part of a decade and it always just works. A few C libraries are not compatible, like mysqldb, but there are alternatives and workarounds. The entire standard library works perfectly as far as I can tell.

Interesting - that's exactly the reason I don't care for gevent as much.

Reply on news.ycombinator.com