Basically a leaky abstraction, you have to think whether your code will block or not.
Go went through the same process, before eventually figuring out that they do need to make their lightweight threads (goroutines) preemptive.
When you're in a "tight loop" (e.g. a matrix multiplication, which is basically 3 nested loops that only load data, do math, write data), Java's virtual threads just won't yield. So if you write your app in the "wrong" way, you lose concurrency.
There's a lot of discussion about this from the Go side. The original issue was this one: runtime: tight loops should be preemptible https://github.com/golang/go/issues/10958
> it's possible to write a tight loop (e.g., a numerical kernel or a spin on an atomic) with no calls or allocation that arbitrarily delays preemption. This can result in arbitrarily long pause times as the GC waits for all goroutines to stop.
The proposed (and ultimately accepted, AFAIK) solution is described here: https://github.com/golang/go/issues/24543
> has put significant effort into prototyping cooperative preemption points in loops, which is one way to solve this problem. However, even sophisticated approaches to this led to unacceptable slow-downs in tight loops (where slow-downs are generally least acceptable).
> I propose that the Go implementation switch to non-cooperative preemption using stack and register maps at (essentially) every instruction. This would allow goroutines to be preempted without explicit preemption checks. This approach will solve the problem of delayed preemption with zero run-time overhead and have side benefits for debugger function calls
I 100% expect Java will have to do through the same evolution. But first they'll probably try to deny reality for a few years. Funny enough, same as has happened with Go and generics.
VertX framework had such a sentinel, but migrating code from the async futures to a normal threadpool can be a bit tedious if your design is poor.
I think what you probably had in mind is that C libraries of the 90s that did M:N threading didn't turn blocking operations into non-blocking?
Using blocking operations to switch contexts is really nothing new. Heck, the cooperative multi-tasking systems of the 80s (Mac, Amiga) all essentially did that for processes (not threads), and so did Unix in the 70s.
Yes, mostly, though my history knowledge is definitely lacking so do correct me if I’m wrong.
But you are right, there was nothing fundamentally missing, probably just no good OS support for non-blocking IO calls in the early days? Though probably the IO-CPU ratio was also different, so the benefits were not as big?
sorry, but Haskell has had it way before Go, with proper STM too. Neither Erlang nor Go are offering the same level of ergonomics for compile-time checked M:N threading.
I'm not very into this Loom virtual threads thing, but... what's the difference between this automagically conversion of blocking into non-blocking in a M:N model and a 1:1 one? I mean, couldn't the same be done with normal threads too?
Loom implements every IO on top of a more modern async OS calls, and these virtual thread context switches are on the order of function calls, so the overhead and number of switches that can happen are much much lower.
Note that this advantage obviously goes away the moment you call into native code. Then the JVM is in the same position as the kernel. It doesn't control the compiler or the stack any more, and so that's why a virtual thread becomes "pinned" at that point and you lose the efficiency (the JVM needs to acquire more kernel threads). Fortunately though the JVM ecosystem doesn't rely on native code all that much, so it should be rare in practice.
This advantage can be brought to other non-Java languages too via Truffle. Truffle languages are reimplemented on top of Java and when programs call into native code they have the option of calling into JIT compiled LLVM bitcode instead of real native code (or indeed any JVM bytecode library). In that situation the JVM remains in control and so things should in theory still be Loom-able. Not sure if that's currently true in practice, but it could be.
Forgive me for staying doubtful, but I recall hearing this same "the JVM can be very fast and efficient because its JIT has complete knowledge and control" spiel back in the 90s, and back then, anyone could clearly see that the JVM was not as fast compared to pre-compiled native code as it was being promised.
> The kernel can't do any of these things - it has to assume a process is a black box that could do anything with its stacks, could be compiled by anything and so on.
The kernel has to assume nothing; it can dictate how userspace processes behave. As an example, a process which plays too many games with its stacks, without kernel cooperation, will quickly find out that signals share the same stack unless the kernel is told to use an alternate stack. A process which uses a register declared in the platform ABI as being for kernel use will find out that it can be unpredictably overwritten on a context switch. There are things like shadow stacks and segment register bases which can only be manipulated when the kernel allows it. And so on.
Of course, for compatibility reasons, the current ABI allows userspace processes to do a lot of unpredictable things, but nothing prevents a new "highly scalable threads" process ABI, with stricter rules, from being developed if necessary. Or it could be that only a few cooperative additions to the userspace to kernel ABI are necessary; we already have things like the many options to the clone() system calls, the futex system call, restartable sequences, etc.
Well head-for-head Java will still lose to C++ in many benchmarks, but that's not really due to compiled code quality, it's more about language semantics. Java is very fast for the sort of language it currently is. The big wins for C++ are that Java doesn't have value types or support for vector operations. Both are under development, actually vector ops is basically done but it's waiting for support for value types (see discussion elsewhere).
Also GCd languages trend towards a functional style without much in-place mutation, whereas C++ trends in the opposite direction, so C++ will sometimes use the CPU cache more effectively just due to prevailing habits amongst programmers.
> The kernel has to assume nothing; it can dictate how userspace processes behave.
Yes in theory you could fuse the language VM with the kernel and research operating systems like MSR Singularity did that. But a normal kernel like NT, Linux or Darwin can't do this and not only for backwards compatibility. The JVM will do things like move a virtual thread stack back and forth from the garbage collected heap and do so on the fly. Unless the kernel contains a JIT compiler, GC and injects lots of runtime code into the app's process it's going to find it tricky to do the same. By the time you've done the same you haven't implemented better kernel threads, you've made the JVM run in the kernel.
Will calling a coroutine do zero heap allocations like async in Rust?
> with the ease-of-use experience of threads
That's highly subjective. Threads usually require locking which is often hard to get performant and correct at the same time. Async/await allows to write concurrent code with no synchronization.
Well you still need some sort of synchronization, because an "await" allows arbitrary other actions to occur. If an await is introduced in code you transitively call then you might find that some invariant you were expecting to hold has now changed across a call when it previously didn't. Fundamentally, locks are about making invariants atomic and that's independent of exactly how code is scheduled and when.
Hmm,I don't see how async/awaits makes a difference. Care to explain?
Like, if you have multiple sources that can add or read from a queue, unless there is a single thread running all your async loops (ala python), you still need some synchronization. At least that's my experience using coroutines heavily in kotlin.
let mut buffer = ... // create buffer for holding data
let mut input: TcpStream = ... // connect to remote endpoint
let mut output: TcpStream = ... // connect to remote endpoint
loop {
select! {
_ = input.readable() => {
input.try_read(&mut buffer)?;
}
_ = output.writable(), if !buffer.is_empty() {
output.try_write(&mut buffer)?;
}
}
}
Synchronization is achieved implicitly by the fact that sequential code executes in only one place at once, so when it executes the reading branch, it does not execute the writing branch. You can apply exactly same reasoning as with any single-threaded, sequential code.
You cannot model this easily with threads. If it was a single thread with blocking I/O, then it could block forever in one branch and stop reacting to events on other branches. If it were multiple threads, then they would somehow need to synchronize accesses explicitly to the shared buffer.