https://tomaka.medium.com/a-look-back-at-asynchronous-rust-d...
It highlights some of the same problems:
- “Just spawn a new task”
- The Send trait isn’t what it means anymore
https://tomaka.medium.com/a-look-back-at-asynchronous-rust-d...
It highlights some of the same problems:
- “Just spawn a new task”
- The Send trait isn’t what it means anymore
I wonder why I haven't hit these issues despite writing web services in Rust for years.
The typical context for stuff like async/await is code that should be running in a separate process so it can block but doesn't. Then it gets ugly fast.
I think perhaps people are trying to push too much into a single process?
That's exactly what's happening. A bit of Erlang/Elixir experience would help a lot of people to create architectures (even in other languages) that do not require such kludges. If your application starts to take on the kind of complexity that you normally deal with a monolithic OS kernel you're doing something terribly wrong.
Reducing scope and simplification, increasing reliability and ultimately aiming for software as a true engineering profession should be our goal. Not fashion and running after certain other eco-systems because they are perceived to be in competition. Rust has promise, it is potentially a game changer and this sort of distraction can be done without. It feels as if over time the Rust project is being hijacked by people that want it to be everything for everybody, and now places itself not only in competition with C (and possibly C++) but also with Node (which is itself in competition with C).
Especially considering project constraints of the Rust project it might pay off to see what ultimately is what makes C++ a problematic language. This can be traced back to Stroustrup dragging in everything and the kitchen sink rather than to stick to his original vision, which was 'C but with classes'. The end result is a giant hairball and fifty (ok, exaggerated) ways of achieving the same thing.
No, that's exactly opposite of what I meant when I described what I'm working on, that all of the alternatives that you both with OP described are terrible in that specific case because of project constraints. I actually measured that!
As to the rest of your comment I fully agree, tired of all the hype cycles myself.
Regarding Bjarne Stroustrup, he wasn't dragged anything, his C++ is described in "The C++ Annotated Referece Manual" and "The Design and Evolution of C++", everything that happened after that is responsability of WG21, where he only has one vote among 300+ persons.
There are several papers from him where he complains about the direction WG21 is going, like "Remember the Vasa".
https://www.theregister.com/2018/06/18/bjarne_stroustrup_c_p...
I positively loved C++ when it was still called 'cfront' and after that it became more and more layers of complexity and tricks. C already had plenty of that (for instance, the weird pointer to a function syntax) and C++ went all out to drag in the various concepts that were present in other programming languages. This wasn't the same as the way say English borrows words and concepts from other languages freely, those are mostly just words and new ways to combinate existing ones. It would be as though English suddenly started using Kanji or logograms, or as if it would start using right-to-left writing or maybe even bottom to top in the middle of 'normal' sentences.
Such inclusivity increases the surface area of the language itself and ultimately makes life harder for everybody using it: you have to learn more because someone else could use these new constructs in their code and you may end up having to interact with it.
I think GvR got that one right with his 'there should be only one way to do it' but even that became a straightjacket.
I'm not saying I have all of the solutions and if Stroustrup wasn't the driving force behind C++'s complexity then I'll be happy to take that back. But I really believe that programming languages should strive for simplicity, just enough to make it all work so that the barrier to entry is small and the amount of gate-keeping can be kept to a minimum. That is the only way to really drive adoption and to hopefully undo some of the crazy fragmentation that we have in the programming language landscape.
The moment the language evolution is driven by some kind of request for improvement, or similar process, with votes on what goes in or not, the outcome is design by commitee.
Many other languages aren't much better than C++, if you check their current versions and the Features/Year rate, including standard libraries.
Also having BDFL only works out as long as it is the original author, afterwards there is no guarantee that the successor has the same vision for the language.
The tragic long term meaning is to either accept complexity, or reboot the programming language ecosystem every couple of decades.
I'll immediately spin that right back at you: people are trying to do too much via too many OS processes. ¯\_(ツ)_/¯
I like my single-OS-process apps that can distribute work internally (via the aforementioned Erlang "green threads"; they are not exactly that but close), or Rust's tokio, or Golang's goroutines and channels and WaitGroups.
I suppose we can all retreat to our corners, shrug and say "well, it's just how I prefer doing things" but I still find the many-OS-processes approach overrated and leaky. You can optimize the program pretty well but then you have to worry about whether the kernel will schedule and distribute work properly (i.e. on a 24-core server, will spawning 24 copies even with CPU core pinning work well?).
I personally prefer to dispense with these worries and just make self-contained apps / services and then be able to put them in any hosting, and deal with scaling issues either by further optimizations, or just bumping the server bill with $20.
The salient different to me is the supervisory structures. In Erlang you must compose them, in operating systems the kernel does that for you, or you move the responsibility into a distributed queue, etc.
As for scale, I don't really know how to reconcile "I'm fine just paying another 20 dollars" with "I'm worried about how my kernel will schedule my processes".
Does it really? I am not an uber Linux expert but I have only heard of systemd doing that. Serious question, I'll appreciate more examples.
> As for scale, I don't really know how to reconcile "I'm fine just paying another 20 dollars" with "I'm worried about how my kernel will schedule my processes".
Sorry if I was unclear, I meant it as "I don't want to worry if the kernel will properly schedule N clones of my single-threaded program so I prefer to write a fully async one in a single OS process and work on it to make it efficient instead; failing that, I'll just bump my hosting so my program has the throughput I require".
It doesn't make much difference though, you can say it's systemd, the kernel, dockerd, or whatever. The point is that operating systems have tons of facilities built in for the management of processes.
> Sorry if I was unclear, I meant it as "I don't want to worry if the kernel will properly schedule N clones of my single-threaded program so I prefer to write a fully async one in a single OS process and work on it to make it efficient instead; failing that, I'll just bump my hosting so my program has the throughput I require".
Why not "I don't want to worry about if my kernel will properly schedule N clones of my single threaded program so I'll just bump my hosting so my program has the throughput I require" ?
Also I didn't suggest single threading, I generally build things as 'async/await' on the inside and 'service oriented' on the outside. So when there's some new domain I don't spawn a new task, I just spawn a separate process.
> Why not "I don't want to worry about if my kernel will properly schedule N clones of my single threaded program so I'll just bump my hosting so my program has the throughput I require" ?
Boring answer: I never prioritized that way of work, I admit. And after gaining tons of experience in several ecosystems and languages I got burned out and right now I can't bring myself to learn systemd properly.
> Also I didn't suggest single threading, I generally build things as 'async/await' on the inside and 'service oriented' on the outside. So when there's some new domain I don't spawn a new task, I just spawn a separate process.
Ah, I see. Fair, I would and have done the same (though that comes with some care when coding the thing so it can determine what it needs to do so spawning the 2nd copy doesn't step on the toes of the first one).
Erlang's supervisor trees are much more powerful than anything the kernel will do for you.
Supervisors are built on `link`, and every process in erlang must always exit with an exit status. This is identical to a process handle with a process exit status.
The whole point of supervisors is that they are extremely simple - you can implement all of Erlang's OTC and supervisory semantics with, more or less, recursive functions (which are the foundation of actors), names, and links.
Joe Armstrong's view of the software world (one with which I strongly identify so forgive me the soapbox mode) is that reliability, not throughput should be our main focus and that failure should be treated as normal rather than as exceptional and that distributed systems are the only viable way to achieve the kind of reliability that we need to bring software engineering into the world. This is a completely different kettle of fish than stringing together a bunch of services using kernel or auxiliary mechanisms, both in terms of results and in terms of overhead.
What you're talking about is the fact that Erlang as a VM is well built. That is irrelevant to whether or not supervisory trees are equivalent to what an OS provides.
> No operating system comes close to delivering that kind of reliability.
Erlang runs on an OS though. Like, Erlang would inherently always be limited by the reliability of the underlying system and in fact relies entirely on it for preemption.
> (one with which I strongly identify so forgive me the soapbox mode)
For the record, I am an extreme fan of Joe's and I routinely re-read his thesis, so you are preaching to the choir in a way.
> This is a completely different kettle of fish than stringing together a bunch of services using kernel or auxiliary mechanisms, both in terms of results and in terms of overhead.
I just totally disagree and in fact you'll find that Erlang's seminal works used the term 'processes' and in fact is based on the entire model of OS processes. Off the top of my head he cites at least two papers about processes and transactions as the foundation for reliability in his thesis.
> Like, Erlang would inherently always be limited by the reliability of the underlying system and in fact relies entirely on it for preemption.
This is factually incorrect, sorry. The Erlang VM uses something called 'reductions' which preempt Erlang processes when their computational slice has been reached which has absolutely nothing to do with the OS preemption mechanism.
And an Erlang system can span multiple machines with ease, even if those machines are located in different physical locations.
Erlang processes are more along the lines of greenthreads than full OS processes but they do not use the 'threads' mechanism the OS provides for the basic scheduling. The VM does use OS threads but this is mostly to decouple IO from the rest of the processing.
Oh, and BEAM/Erlang can run on bare metal if it has to.
Yes, that is true. I don't think that is relevant to the semantics of the constructs.
> The Erlang VM uses something called 'reductions' which preempt Erlang processes when their computational slice has been reached which has absolutely nothing to do with the OS preemption mechanism.
That preemption relies on the runtime yielding at every function call. The only thing that can actually preempt a process mid instruction is the kernel, and actually it can't do that either it's the hardware that yields to the kernel.
> And an Erlang system can span multiple machines with ease, even if those machines are located in different physical locations.
Yes, that's not unique to Erlang, obviously one can launch processes on arbitrary machines.
> Erlang processes are more along the lines of greenthreads than full OS processes but they do not use the 'threads' mechanism the OS provides for the basic scheduling.
I think you're getting hung up on implementation details. The abstraction is semantically equivalent. One might be faster, one might be heavier, one might have some nice APIs, but in terms of supervisory primitives, as I said before, the only thing required is `link` and processes have that.
I'm too lazy to go to the various paper Joe cites but if you take the time you'll find that many of them are about processes