Hacker News Reader

Comment by kaliku2 | Hacker News Reader

while the awaitable task is .. well.. waiting for the query, the thread servicing it is returned to the thread pool to be reused. This helps with thread pool starvation in the case when the process must service many requests.

View on HN

Exactly.

Here's another simple way to think about it: Handling a request usually invokes a couple of layers of your code until you hit some IO - database, file system, network, etc. If that IO is done in a blocking fashion, this request is consuming server resources THE WHOLE TIME, even while it's doing nothing but waiting for a network response. On the other hand, if the IO is non-blocking (async), then this request consumes no resources while it's waiting.

So imagine you have a request that takes 2000ms to service, and 1700ms of that time is spent waiting on the database to fetch some data. Without async, that request will consume 2000ms of CPU time. With async, only 300ms of CPU time is used.

No, the request wont consume 2000ms of CPU time if it's blocking. It will prevent the thread from doing other work which may be bad, but the CPU itself does not have to spend all 1700ms on that thread. At least my CPU doesn't use 100% of a core while waiting for a network package from the database server.

Should have phrased it as "the blocking wait will consume resources (keep a thread busy) for 1700ms whereas the async one will not." Hard to stay both accurate and extremely simple when talking about concurrency.

Right. Or you can phrase this as: using async ... await can't make you wait faster. Waiting will take just as long, but it will be more efficient. The thread is freed up to do other things during the wait, increasing the possible throughput.

So, (is it safe to say?) using await/async will likely be slower for each given awaited operation since there is overhead in thread management and because when the awaited operation finishes it must wait for a thread to become available to continue onward.

There is some overhead in thread management, yes. But in a server scenario (e.g. a web api) there is already thread management happening, as it all runs in threads from a thread pool.

I don't believe that you pay a significant additional cost per await - i.e. if you have one await in your code, then you might as well use as many as you can to chop the operation up into small parts so that operations can flow through better.

If you "must wait for a thread to become available" for any significant amount of time then you have run out of threads despite being async, and you have other problems.

This article talks about the mechanics of how the operation's completion bubbles back up to your code, in some detail: http://blog.stephencleary.com/2013/11/there-is-no-thread.htm...

See also, avoiding some basic mistakes: http://www.anthonysteele.co.uk/AsyncBasicMistakes

in UI code, there is a guideline that "operations that could take over 50ms should be async" (1) so as to not lock the UI thread. But it's clear that this does not apply to web server code where it's all thread pool threads already.

1) http://blog.stephencleary.com/2013/04/ui-guidelines-for-asyn...

The thread overhead is a legitimate concern, but waiting for a thread to be free is usually much much better in the asyn then in the sync scenario. If you have a finite number of worker threads, then async will mean less are pointlessly idling at any time, and thus can handle new incoming requests much faster.

No. When implemented correctly, with async/await you run one thread per CPU core or thereabouts and that thread runs continuously, picking up async tasks when it can, whereas using threads you'll have say 100 threads and each core is picking up threads when it can. And switching threads on a CPU core is a much heavier operation than switching tasks on a thread (you have to switch the whole 4k stack, rather than just whatever's in the task).

It's very possible to stuff it up, to have a conventional thread pool end up competing with your async-handling threads, you can get priority inversions where the OS thread scheduling competes with the task scheduling, and plenty of other pitfalls. But done right async/await should be significantly faster than threading.

Reply on news.ycombinator.com