Hacker News Reader

You are leaving out an important aspect: context switches are expensive, especially with all the mitigations introduced after Spectre et al.

Doing more work in a single thread saves a lot of context switches, especially if IO makes up a large portion of the workload. And even more so with other mechanisms to minimize syscalls, like io_uring on Linux.

Async overhead is highly dependent on the way the async runtime is implemented.

It's true that async requires scheduling and task switching to be implemented in userspace, but assuming that async is used as an alternative to threads, then the OS would be doing a similar amount of work anyway.

The OS may be able to do that more efficiently because it has more information about the system. Or it may be the opposite, because the application has more information about the workloads.

"async = overhead" as a general statement is not correct.

Context switches happen regardless of whether you're using kernel mode ("threads") or user mode ("async") scheduling.

And kernel mode actually has much more efficient and performant context switches, because the kernel can access hardware directly.

(It is sometimes useful to be able to do custom user mode scheduling, but certainly not because of "context switches".)

Do you have a citation for kernel mode having more efficient context switches? What kind of direct hardware access are you referring to that would be better than pushing the register context onto the stack?

In my experience, the exact opposite is true, particularly in the era of CPU mitigations that require TLB flushes upon every kernel-mode context switch.

You're right, kernel-level context switching is much slower than user-level context switching.

User-level can also have the advantage of having more actual context about the task that is running, meaning that it's often able to avoid saving/restoring as much data as a kernel-level switch would. See Go's green threads for a great example of this kind of cooperation between runtime and language.

> Do you have a citation for kernel mode having more efficient context switches? What kind of direct hardware access are you referring to that would be better than pushing the register context onto the stack?

The closest thing to this that I can think of is on 32-bit x86 which did have hardware assisted context switching via TSRs.

As it happens, everybody stopped using it because it was too slow, and a bit painful unless you fully bought into x86's awful segmentation model. Early Linux kernels use it if you want to see it in action.

I don't quite follow your argument there.

This is unrelated to kernel threading.

If you have 1 thread handling 1000 requests with some async io mechanism (epoll, io_uring, ...) ,instead of 1000 threads each handling one request, there are much fewer threads fighting over CPU cores and the 1 thread can stay active much longer, hence reducing the amount of context switches.

Especially with a mechanism like io_uring, which helps minimize syscalls (and hence switching to kernel threads).

False. Both epoll and thread switching use the exact same kernel scheduling mechanisms under the hood.

(When the kernel decides which thread gets to run it's doing the equivalent of an epoll call.)

Thread context switches are about scheduling threads on physical cores. Async is about executing in a scope on a thread. There's no direct conversion here. Packing a thread with async tasks could let you reduce the number of threads you have, which will probably reduce the thread context switching if there is contention for cores. Whether that's significantly better than the cost of async context switches really depends on specifics (even if async context switch cost is near zero, though we're no longer talking about C++ coroutines in that case). But keep in mind: If your threads are mostly waiting, they aren't triggering a lot of context switches. More async means fewer busier threads, less async means more less busy threads. To run into a problem with less async your threads need to be forcing switching at pretty fine-grained level. (BTW, most blocking for IO isn't fine-grained from the perspective of instructions executing on a core) That kind of thing happens, but has solutions other than async.

Asynchronous execution is more efficient if and only if the time spent waiting for a call to complete is longer than the time spent checking if the call has completed plus the time spent dispatching to another asynchronous call (and waiting vs checking if it is completed, and so on).

Remember that concurrency is not parallelism, it's not about the number of threads being used.

> Anyway: don't use async because it's "more efficient".

I don't think efficiency has only one meaning. For instance, you can create millions of coroutines while you're limited to only thousands of threads. Doesn't that mean your program is making better use of your hardware and thus is more efficient?

No, your processor can only do n things at a time, where n is the number of cores (probably like 8) (simplifying hugely of course). It's pretty easy to fill out 8 cores with 1000s of threads, the coroutines only become useful when almost all of them are blocking on a socket or something, periodically switching on and off to handle it.

Yes, but contexts where you're creating a million coroutines are almost certainly I/O bound (like a web server).

Coroutines and threads are not that different.

Threads usually preallocate some stack space, while coroutines usually don't. But that isn't such a big deal.

That's not the only difference. Coroutines also typically don't have nearly as many concurrency hazards because you can control when you yield.

coroutines can be either cooperative or not, depending on implementation, so it depends

And so can threads.

That's an implementation detail.

Pthreads work differently, but historically kernel-mode cooperative threads were also popular at one time.

(We decided to not go there because cooperative parallelism sucks balls.)

Coroutines also don’t need to flush the TLB when they get scheduled. This is a pretty big deal for performance, when you have a lot of them.

Thread context switches don't flush the TLB either. This is an important performance optimisation in the kernel context switch code.

Sure, they can be more efficient in particular situations.

But coroutines have overhead (so do threads). Your millions of coroutines might be worse than your thousands of threads in terms of making efficient use of the cpu cores your app or service has available.

You can actually create millions of threads. Threads are just coroutines managed by the kernel after all.

If you can create millions of threads with some hardware, then you can create billions of coroutines on that hardware. It's just a fact that you can create 2-3 orders of magnitude more coroutines than you can kernel threads on any given hardware.

Agreed in general, that async code is generally less efficient than epoll/kqueue/poll/select/etc. non-blocking I/O. However, note that async code is often quite a bit more efficient than naive Java 1.3-style (pre-NIO) tons-of-blocked-threads code.

Now, io_uring async code is yet another story.

I was working at LimeWire when Apple finally shipped a 1.4 JVM as their system JVM. The whole thing got much lighter weight when we could use non-blocking I/O.

Eh? Async code almost always uses the APIs you describe.

The async runtime is almost always built on top of non-blocking I/O APIs, and the extra layer of abstraction adds overhead. You're rarely calling epoll in your async code, but rather your runtime is handling all of that for you.

That is, unless you're using io_uring, POSIX AIO, etc., in which case your async code might not have the extra layer of abstraction on top of the system APIs.

io_uring isn't typically used directly, you're most likely using liburing which is already an abstraction. I don't really see your point.

But a single thread with async can, and for many tasks, is more efficient that a single thread without async, because you aren’t waiting on requests.

One point also is that until recently most personal computers did not have multiple cores, aka you could think at the lowest level that even multiple threads were just more dangerous async lol

Even making new threads in the JVM isn’t guaranteed to result in more than a single OS thread.

> One point also is that until recently most personal computers did not have multiple cores,

core 2 duo has been released 17 years ago

Dual core started coming out ~2005. It took a few more years for it to dominate the product lineup. Computers tend to have a replacement cycle of ~5 years. Consequently, it's not until about 2012-ish that you could reasonably expect a personal computer to have a multicore computer.

(Yeah, that's still a decade ago, but that's a good deal more recent than 17 years.)

A decade is a long time for computers

There was plenty of async JavaScript code and multithreaded Java code out there in 2013, I promise

There is another use case for async in client-side JavaScript that often gets overlooked: modeling the flow of UI.

So, let's say you need to show a "dialog" which, upon exit, should execute a piece of code (e.g. to show another piece of UI). Maybe that dialog can be invoked from multiple places, and the code to execute (after the dialog) changes accordingly.

You could do that via callbacks, but that is not very composable, and is generally messy.

Better solution: a Promise which gets resolved when the user exits the dialog! Composing multiple pieces of UI then becomes equivalent to composing multiple (async) function calls, you can pass output from one as the input to another, use exceptions to "circuit break" the UI flow, etc.

I'm not convinced of the dialog use-case. Seems rather sketchy to await a user to click OK. What happens when you add another button or two to the dialog?

You return a different result (i.e. the Promise resolves to a different result). The calling code can then decide what to do with it.

In this model, you wouldn't await the user clicking OK, but exiting the modal. The promise only resolves "user took a decision", not the actual choice they took.

The result then contains if the user clicked OK, Cancel, [X] to close the modal or any other state transition that you may have added to that dialog. The recipient (not necessarily the caller that initiated the dialog, in case you chain the responses) can then act accordingly and presumably triggers a different action on OK then the other state transitions.