Rust async/await hits stable tomorrow

Consider that maybe this is because your projects are not the intended targets of async programming.

Async is super not new. Fibers and CSP and etc. are not new. A lot of the benefit of better concurrency is exactly for stuff that scales horizontally, but just because you can scale to hundreds of boxes does not mean you want each box to be bottlenecked on how many connections it can handle. For servers you absolutely want to be able to use your nodes to their fullest. Demand for this is almost certainly driving programming languages.

10 boxes handling 10000 connections > 100 boxes handling 1000 connections (edit: each)

Edit: It's also nice on servers, because even if you have memory for all of those stacks in a no astnc world, you def don't have cache for all of them. Splitting the data that needs to be saved between blocking points from the temporary data that gets you to the next block point (and reusing that memory on the next task) has benefits in a modern memory architecture.

But async programming was always hampered by several issues, which made it's own code generation and syntax necessary. Mostly, that's borrowing over yield points.

This enables Rust to be a language about choice: you can easily mix async and threaded code, with both being supported by the language just as well. The magic is that the `async/.await` support doesn't come with a runtime, you can choose it yourself.

Want predominantly threaded stuff? Use a threadpool (and maybe `await` the result). Predominantly concurrency? Use a runtime like https://async.rs and let run your code on that.

This is the “concurrency vs parallelism” that (e.g.) Rob Pike goes on about[0], rightfully. I think to the uninitiated it can be tough to grasp (not commenting on whether YOU understand or not...). I use it daily, depending on circumstances, w a single-threaded network client that concurrently manages connections to perhaps hundreds of other machines. Most of the wall-time is spent waiting for something to occur on one of the remotes; “waiting in parallel” isn’t going to speed anything up, and nothing happens outside of the relatively infrequent events from the outside network to my client. I can’t forecast what is going to arrive on which socket, so I wait, and process asynchronously. GUIs will operate this way too. The pattern is suitable where you’re in a situation where: “something might happen sometime and I’ll need to do a (quick) bit of work in response”

[0] https://youtu.be/cN_DpYBzKso

I think that there's going to be a new os written in Rust. It will be much lower level than all the current OS. You don't need a lot of protection when the compiler protects you.

The biggest problem with thread pools is that you have a limited number of threads. Either because you set an explicit size limit, or you run out of memory. It's really annoying to debug deadlocks that happen because your thread pool is out of threads. So typically applications that use thread pools just don't use parallelism as much as they would if they could use async/await.

There isn't really any advantage of thread pools. Programming in Go, for example, it's very easy to write async calls and quite awkward to explicitly manage threads. But in practice, there are never really any reasons you miss having thread pools.

Or not. To me, that's like saying "If you need to worry about how to fit your data in memory, you should just move to Hadoop, or something like that instead".

Async/Await is especially suited to Rust though because Rust is all about safety/performance/ergonmics. Async/Await in particular allows eliminating of unsafe blocks that one would have to use to achieve the same level of performance by manually implementing state machines for futures.

Also, generally speaking, I think futures are good, performant abstraction over asynchronous code. It's required a lot of work to get it to stable, and there's still a lot of work to be done to address the multitude of paper-cuts left, but the dividends are huge.

Why Async?

https://rust-lang.github.io/async-book/01_getting_started/02...

PHP/Hack have a model that I think is probably the best overall method for serving HTTP requests in that within the context of a single request, none of the code is multithreaded. I say this is superior because you don't have to worry about race conditions (largely), locking and all that. You might call services that do that but you're insulated from the multithreaded issues of that.

Anyway, knowing that Hack's async-await is strictly cooperative multitasking rather than being multithreaded so you get concurrency and non-blocking I/O without the mental and code cost of threads and locks.

That's why I think it's a big deal.

This is false.

Single-threaded async for the server side was popularized by nginx, which started in 2002. Somewhere in 2007-2009 the Tornado web server showed that even Python could be fast and scalable using this model. In 2009 when node was first released, single-threaded async was already a thing.

What made node interesting was that it came with a pre-existing language where nonblocking code was idiomatic. This removes a lot of the pain of single-threaded async that you see in something like Java where it's easy to accidentally use a blocking call. And speaking of Java, that's also where multithreaded is painful because generally you have mutability, so you really need to know what you're doing to avoid race conditions. At every line where you mutate anything, you have to ask yourself, "what if the thread gets preempted here?" In JavaScript you only have to ask the equivalent question on lines where you see `await`.

Rust's Tokio runtime uses a thread pool. As does C#'s asynchronous runtime. Goroutines are also multiplexed among threads. Asynchronous primitives and multithread programming are not mutually exclusive, Javascript not withstanding.

Interestingly, one language is indeed bucking the trend. Java is introducing fibers (lightweight user-mode threads) that allow traditional (sane, as you call it) code to be efficiently scheduled so that it "performs as well as asynchronous code" without introducing new language primitives. So you'll have the opportunity to see which works best in important mainstream languages. One wonders if Java won't pull off a surprise win here.

> you should think about scalable architecture to be able to scale to multiple machines

No, sorry; that's not a viable approach. CPUs are highly concurrent beasts with dozens of cores. Using these inefficiently with traditional blocking threads is not feasible even if you can spread the inefficiency across "multiple machines." That would simply waste expensive hardware, power, space and cooling. Asynchronous techniques solve real efficiency problems in the real world; it isn't a fad. And you can't fall back to "most projects don't need it" in the context of advising others how to scale large systems; they do indeed "need it" and they will continue to gravitate to programming platforms that provide it while abandoning those that do not.