Coroutines: A Million Stacks (2017)
mikemarcin.com
mikemarcin.com
Yes. With Erlang (or Elixir). Even have them preempt each others, accept connections and respond in a timely manner. If any crash, I they can be restarted without much of an issue. Just saw a few cluster nodes spike to above 1 millions processes recently, then recover, no big deal.
The comment was tongue in cheek, obviously, since it's different type of language and environment completely. But there is one interesting insight I learned and it is about how memory quickly becomes a bottleneck. It maybe be obvious to many here but it's still interesting to see it in practice.
I think the Erlang VM authors have arrived at that conclusion as well. There is a quite complicated allocator involved - it there are multiple carriers (large segments) for different types of memory, inside there are blocks. There are different strategies to fit the allocations (the blocks), different options share or not share free lists between different scheduler threads.
Pretty sure it can. Let's give it a try:
#!/usr/bin/env escript
%%! +P 2000000
%% run with ./spawn_n 1000000
-mode(compile).
main([NStr]) ->
N = list_to_integer(NStr),
T0 = erlang:system_time(millisecond),
Parent = self(),
Waiter = spawn(fun() -> Parent ! {final_result, wait(N, 0)} end),
spawn_n(N, Waiter),
Res = receive {final_result, R} -> R end,
Dt = erlang:system_time(millisecond) - T0,
io:format("Done in ~p sec result:~p~n", [Dt/1000.0, Res]),
ok.
wait(0, Acc) ->
Acc;
wait(N, Acc) ->
receive {result, R} -> ok end,
wait(N - 1, Acc + R).
spawn_n(0, _Waiter) ->
ok;
spawn_n(N, Waiter) ->
spawn(fun() -> Waiter ! {result, 1} end),
spawn_n(N - 1, Waiter).
And it outputs: ./spawn_n 1000000
Done in 1.613 sec result:1000000
In summary, it started 1M processes, all with isolated heaps, pre-emptable, with only a few KB of stack size. They didn't execute much just sent their result to a collector process and then exited. All in about 5x as fast as C++.not useful but maybe interesting if you're into this kind of thing
As much as I like Erlang/Go/Rust/etc., it's not a requirement.
Check also libmill[2] (from the same author), it's more Go'ish:
"It can execute up to 20 million coroutines and 50 million context switches per second."