How much memory do you need in 2024 to run 1M concurrent tasks?

For example, for node, the author puts a million promises into the runtime event loop and uses `Promise.all` to wait for them all.

This is very different from, say, the Go version where the author creates a million goroutines and puts `waitgroup.Done` as a defer call.

While this might be the idiomatic way of concurrency in the respective languages, it does not account for how goroutines are fundamentally different from promises, and how the runtime does things differently. For JS, there's a single event loop. Counting the JS execution threads, the event loop thread and whatever else the runtime uses for async I/O, the execution model is fundamentally different from Go. Go (if not using `GOMAXPROCS`) spawns an OS thread for every physical thread that your machine has, and then uses a userspace scheduler to distribute goroutines to those threads. It may spawn more OS threads to account for OS threads sleeping on syscalls. Although I don't think the runtime will spawn extra threads in this case.

It also depends on what the "concurrent tasks" (I know, concurrency != parallelism) are. Tasks such as reading a file or doing a network call are better done with something like promises, but CPU-bound tasks are better done with goroutines or Node worker_threads. It would be interesting to see how the memory usage changes when doing async I/O vs CPU-bound tasks concurrently in different languages.

The code for several of the languages that are low-memory usage that do the second while the high memory usage results do the first. For example, on my machine the article's go code uses 2.5GB of memory but the following code uses only 124MB. That difference is in-line with the rust results.

  package main
  
  import (
    "os"
    "strconv"
    "sync"
    "time"
  )
  
  func main() {
    numRoutines, _ := strconv.Atoi(os.Args[1])
    var wg sync.WaitGroup
    for i := 0; i < numRoutines; i++ {
      wg.Add(1)
      time.AfterFunc(10*time.Second, wg.Done)
    }
    wg.Wait()
  }

Consider the following code:

package main

import ( "fmt" "os" "strconv" "time" )

func main() {

    numTimers, _ := strconv.Atoi(os.Args[1])

    timerChan := make(chan struct{})

    // Goroutine 1: Schedule timers
    go func() {
        for i := 0; i < numTimers; i++ {
            timer := time.NewTimer(10 * time.Second)
            go func(t *time.Timer) {
                <-t.C
                timerChan <- struct{}{}
            }(timer)
        }
    }()

    // Goroutine 2: Receive and process timer signals
    for i := 0; i < numTimers; i++ {
        <-timerChan
    }

Also for Node it's weird not to have Bun and Deno included. I suppose you can have other runtimes for other languages too.

In the end I think this benchmark is comparing different things and not really useful for anything...

note absolute numbers here: in the worst case, 1M tasks consumed 2.7 GB of RAM, with ~2700 bytes overhead per task. That'd still fit in the cheapest server with room to spare.

My conclusion would be opposite: as long as per-task data is more than a few KB, the memory overhead of task scheduler is negligible.

Let's launch N concurrent tasks, where each task waits for 10 seconds and then the program exists after all tasks finish. The number of tasks is controlled by the command line argument.

Leaving aside semantics like "since the tasks aren't specified as doing anything with side effects, the compiler can remove them as dead code", all you really need here is a timer and a continuation for each "task" -- i.e 24 bytes on most platforms. Allowing for allocation overhead and a data structure to manage all the timers efficiently, you might use as much as double that; with some tricks (e.g. function pointer compression) you could get it down to half that.

Eyeballing the graph, it looks like the winner is around 200MB for 1M concurrent tasks, so about 4x worse than a reasonably efficient but not heavily optimized implementation would be.

I have no idea what Go is doing to get 2500 bytes per task.

Well, if it isn't the classic unwavering confidence that an artificial "hello world"-like benchmark is in any way representative of real world programs.

I urge anyone making decisions from looking at these graphs to run this benchmark themselves and add two things:

- Add at least the most minimal real world task inside of these function bodies to get a better feel for how the languages use memory

- Measure the duration in addition to the memory to get a feel for the difference in scheduling between the languages

Am I wrong?

UPDATE: From the replies below, it looks like I was right about "no concurrency takes place", but I was wrong about how long it takes, because `tokio::time::sleep()` keeps track of when the future was created, (ie when `sleep()` was called) instead of when the future is first `.await`ed (which was my unsaid assumption).

Go won because it served a need felt by many programmers: a garbage-collected language which compiled to native code, with robust libraries supported by a large corp.

With Native AOT, C# is walking into the same space. With arguably better library selection, equivalent performance, and native code compilation. And a much more powerful, well-thought-out language - at a slight complexity cost. If you're starting a project today (with the luxury of choosing a language), you should give C# + NativeAOT a consideration.

    import kotlin.time.Duration.Companion.milliseconds
    import kotlin.time.measureTime
    import kotlinx.coroutines.async
    import kotlinx.coroutines.awaitAll
    import kotlinx.coroutines.coroutineScope
    import kotlinx.coroutines.delay
    
    suspend fun main() {
        measureTime {
            coroutineScope {
                (0..1000000).map {
                    async {
                        delay(1.milliseconds)
                    }
                }.awaitAll()
            }
        }.let { t ->
            println("Took $t")
            val runtime = Runtime.getRuntime()
    
            val maxHeapSize = runtime.maxMemory() 
            val allocatedHeapSize = runtime.totalMemory()
            val freeHeapSize = runtime.freeMemory()
    
            println("Max Heap: ${maxHeapSize / 1024 / 1024} MB")
            println("Allocated Heap: ${allocatedHeapSize / 1024 / 1024} MB")
            println("Free Heap: ${freeHeapSize / 1024 / 1024} MB")
        }
    }

   Took 1.597011084s
   Max Heap: 4096 MB
   Allocated Heap: 2238 MB
   Free Heap: 1548 MB

This is the code:

  # /// script
  # requires-python = ">=3.12"
  # dependencies = ["uvloop"]
  # ///
  
  import asyncio
  import sys
  
  import uvloop
  
  
  async def main(num_tasks):
      tasks = []
  
      for task_id in range(num_tasks):
          tasks.append(asyncio.sleep(10))
  
      await asyncio.gather(*tasks)
  
  
  if __name__ == "__main__":
      num_tasks = int(sys.argv[1])
      # uvloop.run(main(num_tasks))
      asyncio.run(main(num_tasks))

  /usr/bin/time -l -p -h uv run async-memory.py 100000

  170835968  maximum resident set size

  204259328  maximum resident set size

[0]: https://github.com/MagicStack/uvloop/

I feel this is so misleading. For example, by default after spawning, Erlang would have some memory preallocated for each process, so they don't need to ask the operation system for new allocations (and if you want to shrink it, you call hibernate).

Do something more real, like message passing with one million processes or websockets. Or 1M tcp connections. Because, the moment you send messages, here is when the magic happens (and memory would grow, the delay when each message is processed would be different in different languages).

Oh, and btw, if you want to do THAT in erlang, use timer:apply_after(Time, Module, Function, Arguments). Which would not spawn an erlang process, just would put the task to the timer scheduling table.

And Elixir was in the old article, and they implemented it all wrong. Sad.

I believe each task in Julia has its own stack, so this makes sense. Still, it does mean you've got to take account of ~16 KB of memory per running task which is not great.

The rust code is really checking how big Tokio's structures that track timers are. Solving the problem in a fully degenerate manner, the following code runs correct correctly and uses only 35MB peak. 35 bytes per future seems pretty small. 1 billion futures was ~14GB and ran fine.

    #[tokio::main]
    async fn main() {
        let sleep = SleepUntil {
            end: Instant::now() + Duration::from_secs(10),
        };
        let timers: Vec<_> = iter::repeat_n(sleep, 1_000_000_0).collect();
        for sleep in timers {
            sleep.await;
        }
    }

    #[derive(Clone)]
    struct SleepUntil {
        end: Instant,
    }

    impl Future for SleepUntil {
        type Output = ();

        fn poll(self: Pin<&mut Self>, cx: &mut Context) -> Poll<Self::Output> {
            if Instant::now() >= self.end {
                Poll::Ready(())
            } else {
                cx.waker().wake_by_ref();
                Poll::Pending
            }
        }
    }

The point I'm making here is that synthetic benchmarks often measure something which doesn't help much. While the above is really degenerate, it shares the same problems as the article's code (it just leans into problems much harder).

Rust's `join_all` uses `FuturesUnordered` behind the scenes, which is pretty intelligent in terms of keeping track of which tasks are ready to make progress, but it does not use tokio/async_std for scheduling. AFAICT the only thing being measured about tokio/async_std is the heap size of their `sleep` implementations.

I'd be very interesting in seeing how Tokio's actual scheduler performs. The two ways to do that are:

- using https://docs.rs/tokio/latest/tokio/task/join_set/struct.Join... to spawn all the futures and then await them

- spawn each future in the global scheduler, and then await the JoinHandles using the for loop from the appendix

As other commenters have noted, calling `sleep` only constructs a state machine. So the Appendix isn't actually concurrent. Again, you need to either put those state machines into the tokio/async_std schedulers with `spawn`, or combine the state machines with `FuturesUnordered`.

This is a sample of 1 usecase, (so questionable real-worldness) but the difference is really eye-opening. Congrats to the C# team!

The cost wouldn't be just Memory because the network card and CPU also enter the game.

Note that Go and Java code are not doing the same! See xargon7 comment.

    async function main() {
        const numTasks = parseInt(process.argv[2], 10);
        const taskDuration = 10000; // 10 seconds

        const tasks = Array.from({ length: numTasks }, () =>
            new Promise(resolve => setTimeout(resolve, taskDuration))
        );

        await Promise.all(tasks); // Wait for all tasks to resolve
        console.log("All tasks completed.");
    }

    main().catch(err => {
        console.error("Error occurred:", err);
    });

  SCO: "thread creation is about a thousand times faster than on native Linux"

  Talk is cheap. Show me the code.

https://lkml.org/lkml/2000/8/26/52

Souce: https://github.com/microsoft/referencesource/blob/master/msc...

https://learn.microsoft.com/en-us/dotnet/core/deploying/nati...

Note 1: The gist is in Ukrainian, and the blog post by Steve does a much better job, but hopefully you will find this useful. Feel free to replicate the results and post them.

Note 2: The absolute numbers do not necessarily imply good/bad. Both Go and BEAM focus on userspace scheduling and its fairness. Stackful coroutines have their own advantages. I think where the blog post's data is most relevant is understanding the advantages of stackless coroutines when it comes to "highly granular" concurrency - dispatching concurrent requests, fanning out to process many small items at once, etc. In any case, I did not expect sibling comments to go onto praising Node.js, is it really that surprising for event loop based concurrency? :)

Also, if you are an Elixir aficionado and were impressed by C#'s numbers - know that they translate ~1:1 to F# now that it has task CE, just sayin'.

Here's how the program looks in F#:

    open System
    open System.Threading.Tasks

    let argv = Environment.GetCommandLineArgs()

    [1..int argv[1]]
    |> Seq.map (fun _ -> Task.Delay(TimeSpan.FromSeconds 10.0))
    |> Task.WaitAll

For me Python worked surprisingly well, while Go was surprisingly high on memory consumption.

https://news.ycombinator.com/item?id=41827288

List<Task> tasks = new List<Task>(numTasks);

Also some other languages seem to be missrepresented.

Seems like someone had good intentions but no idea about the languages he tried to compare and the result is this article.

I must be missing something - isn’t Go supposed to be memory efficient? Perhaps promises and goroutines aren’t comparable?