> but has other advantages, like reduced resource consumption
Could you expand on that? I've never heard that mentioned about async before.
Could you expand on that? I've never heard that mentioned about async before.
Now you do this 1000 times you use barely any resources but the server uses 1GB in stack allocations which might be the maximum and noone else can connect.
Yes, a thread pool, consisting of one thread per core/computing unit. The units of work are then scheduled between the threads. Units of work here being some kind of IO, e.g. servicing HTTP requests.
> but if you want to spin up a few hundred thousand of them...
Hm. Thought there was a limit for work that can be done concurrently by the CPU, based on the number of cores/hyper-threads available. Found this on threads and IO performance [1], it seems to make the same point.
What kind of work load is common to spread over so many threads (on the same machine)? Does the OS switch efficiently between hundred of threads on regular CPUs? Genuinely interested.
1: https://www.jstorimer.com/blogs/workingwithcode/7970125-how-...
> Thought there was a limit for work that can be done concurrently by the CPU,
Right. But in an IO bound scenario, the CPU isn't doing work; it's waiting on IO. So, because threads are generally heavy, you don't want a ton of them, taking up memory, doing nothing.
But, when you have lightweight threads, you can spin up one per connection. This ends up being simpler, and you don't have the large memory usage. This is what nginx does, in a sense. It still has a worker per core, but each of those workers can handle thousands of requests simultaneously, because it's all non-blocking.
That limit to concurrent work is exactly why non-blocking architectures are so important, and task systems fit into them really nicely.
(I also misinterpreted the context as I read the top level comment as async vs a single thread with blocking code, but after rereading it that makes more sense.)