I'm not feeling the async pressure
lucumr.pocoo.org
lucumr.pocoo.org
All .NET async APIs take an optional cancellation token parameter which solves his flow-control problem by allowing the async request to be canceled at any time. If the token is canceled, the async task will (or should) throw an OperationCanceledException which can then be cleanly handled in a standard try/catch block up the stack.
The best part about this is that it pervades the entire .NET runtime, the APIs, code examples, and has excellent documentation on correct usage patterns. Sure, a 3rd party library could choose not to support cancellation tokens, but they would be going against the entire .NET ecosystem by doing so. Every other async implementation I've seen has really seemed like a haphazard bolt-on.
I honestly don't know how I would write robust async code without cancellation tokens, so I guess he has a point when it comes to javascript and python ecosystems.
https://developer.mozilla.org/en-US/docs/Web/API/AbortContro...
I've used both .NET async/await (4 years ago) and Go ever since. The semantics and feel is nearly the same, the look is different.
I'm not sure how it would solve backpressure though, because canceling a task is normally done by the caller, while backpressure comes from the other direction of the call stack.
I’m not sure I agree with the writer that async should handle back pressure. To me that should be handled more explicitly by something purpose built, e.g .NETs Channel type.
It's something that I'm both surprised and not all that surprised it hasn't made its way into more async/await languages exactly because of that confusing complexity.
I believe that in Python, due to its explicit over implicit philosophy, you could almost fake SynchronizationContext/Scheduler dynamics with a smart enough "event loop" as your outermost async runner, monkey patching coroutine return Futures/Tasks to include CurrentSynchronizationContext/CurrentScheduler information. It even looks like at a shallow skim some of that is designed into the default asyncio event loop system in the form of "custom event loop policies", but in the same skim it seems like its both more complex in its own way than SynchronizationContext/Scheduler (such as its primary job seems to be in handling platform specifics), and cursory searches don't show anyone attempting to use it for backpressure management.
Not that in most cases I would write backpressure management code directly in SynchronizationContext/Scheduler "magic" today: I'd use ReactiveX glue and explicitly define Schedulers there and/or make use of "off-the-shelf" RX backpressure combinators. But the low level capability at the .NET SynchronizationContext/Scheduler level is an often overlooked "magic" to async/await that can get results from existing async/await code without needing to rewrite existing async/await code. (I think the fact that RX.NET itself can piggy back off of that low level most directly as opposed to in other languages/environments implementing its scheduler support and backpressure work with less low level underlying support is also easy to overlook that .NET seems to be doing it all more cohesively under the hood.)
This statement is completely false. The .NET runtime is inundated with bad code examples resulting in deadlocks with async code.
In my professional experience, the WriteableBitmap class is completely unusable. All official code samples exhibit the same brokenness.
They all tell you to do the same thing: lock() on the WriteableBitmap object, get the buffer, perform an update the buffer, unlock() on the WriteableBitmap object. But you try to lock() on a backgroundworker (the sane thing to do) and it fails because lock() can only be called from the UI thread. So you lock() on the UI thread, async into a backgroundworker (the second most sane thing to do) to update the buffer, and boom, WPF deadlocks. The official Microsoft UI framework uses .Lock(), not .TryLock(), and you are deadlocked forever. There is no solution to this problem. No solution exists. Abandon all hope ye who enter here.
C#/.NET+async is horrible. Maybe this is the best async implementation that's been done, but if that's the case, the best implementation is worse than starting a new thread every time you need to block on IO. The only way to write good async C#/.NET code is to not write async C#/.NET code. But guess what? As you said, C#/.NET is inundated with async APIs, and it falls down every time you blink at it.
C++ is looking to standardize async/await into the language. I think I'd rather pick up growing corn.
https://docs.microsoft.com/en-us/dotnet/api/system.windows.m...
I can't see your code, but this sounds more like a misunderstanding of the requirements around a UI thread and the task scheduler than any issue with async.
https://medium.com/rubrikkgroup/understanding-async-avoiding...
And the below seems to be a correct implementation of your desired functionality.
https://stackoverflow.com/questions/9868929/how-to-edit-a-wr...
Async .net may have some pitfalls but it is overall versatile which was the original sentiment anyways...
Second, the "seems to be correct" article you linked is what we did. It's a sensible, logical solution to the problem. Which is why we wrote it that way to begin with. Hell, we might have copied it directly from that SO answer.
It doesn't work. It deadlocks. You call out to the UI thread dispatcher, lock the object, then go back to doing stuff on a background thread. Some time later, the UI thread is updating the UI, and it tries to update your element, and it tries to lock on the lock object. (it can't, because it's already locked) Some time later, the background task is done. It schedules a task on the UI thread to unlock the object. There's a problem though: the UI thread is locked on a mutex. A single hardware thread cannot wait on both a mutex and the task queue at the same time. Deadlock.
Just because it's the highest voted answer on SO doesn't mean that it isn't completely wrong.
The second highest voted answer states that what the questioner is asking for is impossible. This answer is the correct one. Our least bad solution to this problem is to update a buffer owned by the background thread, and then use the UI thread to copy pixels from the background thread's buffer to the UI thread's buffer. Lock, copy the buffer, unlock, without ever relinquishing control of the UI thread. This solution is not ideal from a performance standpoint, but at least it doesn't deadlock.
C#/.NET's async is fine if you're not doing anything more complicated than querying a database or paying data around on a socket. It falls down really hard if you start tickling its underbelly.
That’s just a leaky abstraction. Updating image from background thread is a sane thing to do from general-purpose programmer POV. To understand why it’s not so good idea, and why it’s not supported, you need to know what happens under the hood. Specifically, how 3D GPU hardware works and executes these commands.
> No solution exists.
From graphic programmers POV, the sane solution — only call GPU from a single thread. In D3D11 there’re things which can be called from background threads. It’s possible to create new texture on the background thread uploading the new data to VRAM, pass the texture to the GUI thread, then on the GUI thread either copy between textures (very fast) or replace textures destroying the old one (even faster). Unfortunately, doing so is slower in many practical use cases, like updating that bitmap at 60Hz: creating new resources is relatively expensive, more so than updating preexisting one.
Then there came async frameworks a la Node, Twisted et al. and changed everything. Again in my opinion, async code is harder to reason about versus synchronous code.
Things to keep in mind:
- Are you really working at scale? Does the arguably added complexity of async benefit your particular use case? Specially when SPA technologies allow to build simpler backend for frontend systems (pure API, no HTML rendering). And not only regarding pure operational performance, Rails is still impossibly hard to beat when it comes to productivity.
- New players like Go, Rust have async capabilities but you dont necessarily need to use them to perform closer to native speed, hence becoming simpler solutions than Node, Ruby, or Python. Guess that also applies to old dogs with new tricks in the JVM (Micronaut, Quarkus...)
One change here which has also made this less important is the trend toward serverless/microservice architecture. In a world where you were running your MVP on a single VPS instance somewhere, and you were running a monolithic webserver which handled everything from request parsing to business logic, then moving from one-thread-per-request to a runloop implementation represented a potentially huge win in terms of performance.
But nowadays we largely work with more atomized bits of logic: business logic is implemented in terms of small, self-contained operations, and the problems of scale are delegated to some other system or orchestration framework. In that world it matters much less if your database access is blocking a thread or not, because throughput issues can be solved at a different level.
I disagree.
Although it supported (cooperative) multithreading, Windows 3.1 applications typically had one thread that ran a message loop, invariably written in C or C++. Same with MacOS applications since 1984, except at first it was Pascal and 68000 assembly. CICS was originally single-threaded, running transactions written in System/370 assembler and later COBOL, until they bolted multithreading onto it in order to take advantage of the new SMP mainframes IBM was building. thttpd is from 1998 and is of course written in C; see http://www.acme.com/software/thttpd/benchmarks.html. One of the biggest changes in web-server software over the last ten years has been the move away from Apache to nginx, which is of course event-driven and written in C.
> async code is harder to reason about versus synchronous code.
Well, this is surely true. But generally the choice isn't async code or sync code; it's async code or multithreading. And async code is much easier to reason about than threads-and-locks code.
When you can hack it, a better approach is multiprocessing, also known as message passing, where each thread has its own private memory space, so it looks just like sync code. That's how Unix was built (though not just one but several broken threads models got bolted onto it later), it's how Erlang works, and it's the model adopted for JS in Web Workers. It has some performance cost. (Transactions, as in CICS or the Haskell STM, are another appealing option.)
Rust offers a really interesting alternative here: it statically proves that it's impossible for multiple threads to share access to writable memory (except when you cross you fingers and say "unsafe"), so in effect the threads are compiler-enforced multiple processes, but with the ability to dynamically freeze data so you can start sharing it between those processes.
From my point of view, what happened is that preemptive shared-memory multithreading with locks became a lot more popular in the 1990s and 2000s, driven by a rise in popularity of two systems originally developed for microcomputers without memory protection: Win16 (later Win32, Win64) and Java. Real-time control systems like the THERAC-25 control system have always used preemptive shared-memory multithreading, invariably with locks. (In the minimal case they use interrupt handlers rather than full-fledged multiple threads, but those have the same issue.) With the advantage of historical perspective, we can now see that it was a mistake to extend that programming paradigm to things like GUIs and web servers, but to some extent we're stuck with it for legacy systems like the JVM, Win64, and pthreads.
For new systems, though, we can use async, or multiprocessing, or transactions, or Rust's borrow checker, or a mixture.
> But generally the choice isn't async code or sync code; it's async code or multithreading.
So true! years ago I worked with CORBA on top of ACE's libReactor and the result had an interesting mixture of the worst of the worlds of async and threads-and-locks ;)
As for web development, JS has been async since day 1. For a little while Opera was multithreading it, but that was really hard to program on, because they didn't even offer locks. And Perl was not normally multithreaded, but before POE it wasn't async either, but rather multiprocess. msql was async, as was FastCGI, originally. So I'm not sure I agree even within the scope of webdev history. Maybe you got started late when lots of people were already using Java?
For example we ran a lot of Rails code on top of FastCGI with lighttpd but as far as I can remember the Rails code we wrote wasnt affected by the fact that FastCGI was async.
It seems advisable to begin that response now. The linked article might be the beginning of such a response, but it seems too tentative. We may need an Iron Law of Flow Control, visibly acknowledged and observed in each system that uses an async/await facility, at the point of use, or a note explaining where it is handled farther back up the chain.
TCP vs IP is an excellent example of such an alternative: IP does not bother with buffering, except as a completely local performance optimization, and happily drops packets at the first hint of trouble, assured that somebody closer to the source has buffered copies of whatever they actually care about.
Did you mean UDP? But even if so, this description overall seems way off in the weeds.
The parent describes it quite accurately: IP makes no guarantee on delivery and allows dropping packets, TCP promises to fix that on a higher level for applications that need it.
I mean to suggest that buffering isn't a proper feature distinction between TCP and unreliable protocols.
Separately, it seems a tad obscure to use IP as the exemplar here--doesn't most everyone want UDP features on top?
Like what? UDP is lightweight and bare-bones, no?
Normally, if you need a checksum, you need one that cannot be forged, and you need more than 16 bits. So the L3 one is just overhead.
I have never seen an actual use of UDP that looked at the port numbers, although firewalls often use them for filtering.
If IPv6 disallows simpy zeroing the field, that sounds like it's not just a nuisance.
> I have never seen an actual use of UDP that looked at the port numbers, although firewalls often use them for filtering.
I don't know what you're saying here. UDP's ports model isn't optional in any sense, it's just like TCP's.
If you run several UDP-based services on your server, port numbers are how the server knows which packets are destined for which processes.
But the checksum is very anemic, inadequate when correctness matters.
To be clear: every intermediate router is expected to implement only IP, and completely ignore UDP or TCP. Each discards packets freely.
(Router buyers have taken up violating the design, and meddling with TCP along the route. Yet, the point stands.)
The buffer the original comment is talking about is likely the TCP receive buffer, which is used both to allow the application layer to process bytes at its leisure rather than as they arrive, and to allow out-of-order data to be reassembled correctly rather than retransmitted in-order every time.
The receive buffer is documented at the protocol level. Well, honestly I don't recall if the buffer is ever documented in the spec or just common implementation practice. There is a documented "receive window" though, which is how many bytes the receiver is comfortable having "in flight" - sent but not acknowledged. The existence of this window implies the existence of a buffer at least as large as that window on the receiver.
I think that there is value in async/await as a concept, but that some language's specific implementations will be eventually seen as a mistake. My primary experience is in .Net, and I've grown disenchanted with async/await in that realm. I feel like it makes the easy things easier and the hard things harder[1], which, IMO, is the wrong trade-off for a concurrency primitive[2]. By comparison, what I've read about async/await in Rust seems very promising.
[1] Specifically, I think that the Task API is warty and over-complicated, with abominations such as .ConfigureAwait (I know that there are situations where it can be useful, but those don't make the method any less terrible). While await doesn't strictly depend on Task[1a], in practice you're going to be using Task.
[1a] http://blog.i3arnon.com/2018/01/02/task-enumerable-awaiter/
[2] Before anyone chimes in with "async/await has nothing to do with concurrency/threads---it's just state machines": yes, that is technically true in the narrow sense of how the compiler desugars the async/await keywords. However, as soon as you start talking about actually scheduling and executing your tasks, concurrency/threads become very important.
The task library (TPL) is a smidgen wordy, as was MSFT's way about ten years back, most folks will never get deep into it though.
Need an easy way to cancel 100 async requests? Easy, just pass in a cancellation token and cancel it when needed. All the requests will throw an OperationCanceledException which can be cleanly handled.
I haven't seen anything this nice in javascript or python.
I mean... I get it---it's great how Microsoft put in an effort to add async equivalents to most of your standard operations. But my complaint is about how almost all of those async methods are built upon Task/Task<T> (which, IMO, is a flawed foundation).
> There's also always (from what I've seen) an optional cancellation token parameter that allows an easy mechanism to alleviate backpressure.
> Need an easy way to cancel 100 async requests? Easy, just pass in a cancellation token and cancel it when needed. All the requests will throw an OperationCanceledException which can be cleanly handled.
No significant disagreement from me there. It is nice that most core async methods support (or try to support) cancellation.
Do you know any not-flawed implementation?
As far as .Net goes, I've not looked around. I feel like it would be somewhat of a fool's errand to develop an alternative async/await implementation because almost all of the core APIs use Task/Task<T>, so you'd either be using Task/Task<T> anyway or re-producing a lot of APIs.
I like them, very flexible. It’s easy to implement these tasks on top of pretty much anything.
Here’s how I have implemented back pressure when I needed something similar: https://github.com/Const-me/pCloud/blob/master/pCloudCore/Ut... That class is very generic, knows nothing about the nature of the tasks, yet it’s able to do the right thing. Specifically, it limits count of pending tasks to a small number (8 by default), when exceeded the `post` method will start to “block” in the async way, that’s why it returns `Task<Task<TResult>>` instead of just `Task<TResult>`.
Threads provide a very privileged form of state machine that are tightly coupled with the underlying platform. This means you have to be thinking about things like cache coherency and context switches. Without careful use of other synchronization primitives (or stronger language guarantees[1]) it's easy to find yourself writing incorrect or less-than-performant[2] code.
[1] Like in Rust
[2] e.g. creating bottlenecks by blocking on access to a shared resource (Edit: perhaps a better way to word that would be "accidentally reintroducing blocking behavior by using synchronization primitives to access a resource")
But even in threaded code you can have buffering problems. For example, Linux' default socket buffer size sysctls are arguably far too large, optimized for high throughput of a few connections (i.e. benchmarks), not for good responsiveness. That makes it too easy for any massively concurrent service daemon, threaded or not, to flood the outbound network queues.
The answer for any I/O model is to never, ever implement an in-memory input or output sink with an unlimited size, whether for buffering data, connections, or any other resource (e.g. stack frames if an event loop recursively reenters itself). That solves most of the problems right there. The remaining problems are more nuanced, and may require reducing buffer sizes as much as possible (think single-digit Kb instead of Mb[1]), and considering how various resource buffers and APIs interact (additively, multiplicatively) to create hidden buffer bloat.
[1] Especially if you're not doing any heavy processing of the buffer contents, such as audio-video processing where you may need moderately larger buffer sizes to benefit from pipelined and SIMD processing.
Not all of them. Go has Goroutines. And Java is settling on virtual threads (fibers) with Project Loom instead of async/await.
yup. Back pressure doesn’t compose in the world of callbacks / async. It does compose if designed well in coroutine world (see: erlang).
> async/await is great but it encourages writing stuff that will behave catastrophically when overloaded.
yup. It’s very hard, in larger systems impossible, to do back pressure right with callbacks / async programming model.
This is how I assess software projects I look at. How fast is database is one thing. What does it do when I send it 2GiB of requests not reading responses? What happens when I open a bazillion connections to it? Will a previously established connections have priority over handling new connections?
For a javascript example I ran into recently, say I am firing off a fetch for each image that comes into view in a large gallery. If I suddenly scroll down to the 1000th image, a naive implementation might fire off 1000 fetches for all the images we scrolled past. Then you'll be waiting a long time before the images in your current viewport is loaded.
Backpressure can save you a little bit here. Say you do the semaphore trick mentioned in the article and only allow a max of say 10 fetches in flight at once. Then if you quickly scroll through, all the subsequent fetches after the initial should fail, including the ones at the viewport you stop at. But since the queue is short, when the images in your current viewport retries it should now succeed.
This works but it isn't ideal. Ideally I would be able to just reprioritize the newer fetches to be LIFO instead of FIFO. Or maybe inspect what's currently queued up (and how big the queue is) so I can cancel everything that I don't need.
The backpressure solutions might just be a symptom of async tasks not being controllable in any way once started which is why you're forced to commit to it or not from the start even if that might not be the best point in time to make that decision.
I truly believe BEAM's runtime has a million good ideas in this space that we should be stealing!
https://news.ycombinator.com/item?id=21932726 said exactly what I was typing out on how to handle this general situation without the upcoming API for the exact use case mentioned in this comment https://news.ycombinator.com/item?id=21932328
50ms is way too short, a user on a 1080p screen would need to scroll at 360 pixels per second to avoid loading every image along the way. Even trying to do that on purpose is hard as hell https://clickspeedtest.com/scroll-test.html. Guaranteed that every mobile user is going to be loading every image on scroll over 4g as well creating the original back pressure issue now with 3 more frames of lag.
"smooth" does not mean "check every 3 frames" it means schedule things in a way they can happen immediately when the browser is ready, not your timer.
You could tie it to an "on-scroll" type of event, like how many text boxes will fire an event X milliseconds after a text edit event, so that they can process the text when the user is not likely to be typing.
A single timer by itself won't busy-wait, but I agree that running a timer all the time is a waste. Just run it when scrolling, when CPU use will already be high.
You can even reuse the same data structure and just flip the behavior under load, what's important is having FIFO behavior normally and then LIFO behavior when congestion occurs.
So no, browsers don't have these problems solved out of the box. And it seems like what they are implementing will suffer from the exact problem I described here too.
I think you might also be thinking of bad implementations of infinite scroll which isn't what I am talking about. Have you never use something like photos.google.com? Scrubbing/scrolling to an arbitrary point in time is honestly a great user experience. Would you rather wait hours/days for a webpage to load all the past photos images you've ever taken? Or have to deal with pagination and click through hundreds of pages to find what you want?
Separating these two operations allows code to use multiple write() calls to build up a single record atomically before yielding control to the system, where some other task might also write to the same stream. This reasoning is only valid if the program is running on a single thread, but that’s a reasonable architecture decision for many programs.
As an example, there's a big difference between the async needs of one paying customer who's loading one medical chart web page vs. some free third-party web crawler trying a daily full text scan of an entire site.
Async with cost functions feels like a promising area for real-world use cases.
In any case all this can be solved without major rewrites by making sure every awaitable is awaited at some point in the future. Instead of doing this:
while connection.accept():
handle_connection()
connection_pool = []
while connection.accept():
connection_pool.append(handle_connection())
if len(connection_pool)>=MAX_CONNECTIONS:
await wait_any(connection_pool)
And in fact in python -- and other programming languages too I bet -- you get a warning on exit if there are any awaitables that never got awaited. In my opinion that should be an error. I can't think of a single scenario where it'd be proper form to never await an awaitable. You might await immediately, sometime in the future, or at the end of the program. But you never don't await at all.
Threading isn't any easer, or harder. If you spawn a thread for each connection you run into the same issue as await unless you do something about it. If you use a pool of threads you get backpressure for free, but the same goes for a pool of awaitables!
tl;dr: async/await has the exact same problems as threads, the tooling around async/await is just less mature. Rewriting async code to provide back pressure is near trivial.
Async Python gives you the ability to set up functions that act upon multiple stdout/stderr streams and process them in real time.