Hacker News Reader

Comment by piokoch | Hacker News Reader

While this is definitely a good new, we need to be very careful. Why? Those new fancy threads are stored on heap and need to be garbage collected. If you buy into ads and believe that you can create for free millions of threads, then, well, it is not gonna work on production.

Because of this server based on new threads does not need to be more performant, Jetty server guys tested this and they were not that happy: https://webtide.com/do-looms-claims-stack-up-part-1/

Before getting too excited I advise to watch Tomasz Nurkiewicz lecture on the subject - https://www.youtube.com/watch?v=n_XRUljffu0, it explains what are the trade-offs here.

No silver bullet, again.

View on HN

1. Platform threads place a heavier burden on the GC. It's true that virtual threads are allocated on the heap, but platform threads are GC roots, which is worse. The GC easily deals with a gazillion heap objects; it's rather unhappy with lots of roots. The number of heap objects that virtual threads occupy is roughly the same as the number of heap objects that async code allocates anyway.

2. The Jetty experiment measured the wrong thing as they misunderstood the origin of the "million thread" scenario. What happens in a real application is that you have some number of threads with deep stacks servicing incoming requests -- say 50K concurrent sessions -- and then each of those fans out to, say, 19 micro services in parallel, each of those outgoing requests is done on a virtual thread with a very small stack, and that's how you get to 1M threads. I.e. when you have a high number of threads, only a small minority of them (5% in this example) have a deep stack.

3. I don't think anyone would claim anything is a silver bullet. All virtual threads do is let a server service the same throughput as asynchronous code does, but the code is much simpler and it is observable, i.e. easily debuggable and profitable, something that async code can't do.

Regarding #1, would not the stacks of the lightweight threads have to root have to root any object on it? Otherwise the GC would free objects out from under the virtual thread, right?

I could imagine that by having fewer physical threads running, the stop-the-world part of garbage collection could suspend the runtime more quickly. That could reduce the effect of GC-pauses.

Virtual thread stacks reference the objects that local variables on the stack reference, but they are not themselves GC roots. GC roots are special objects that the GC starts its scan of the heap from, and they tend to be particularly costly, at least for most of OpenJDK's GCs. Virtual threads are just ordinary heap objects that can reference other objects.

> I could imagine that by having fewer physical threads running, the stop-the-world part of garbage collection could suspend the runtime more quickly. That could reduce the effect of GC-pauses.

Precisely. Although it's worth mentioning that while that's true for G1, ZGC does not stop-the-world when scanning roots, including platform thread stacks (https://openjdk.org/jeps/376).

What is so costly about GC roots? Considering the stack is used in a LIFO manner, most stack pages never change.

They have to be scanned in every collection and G1 scans them in a stop-the-world pause. Other references may not be scanned at all in most collections (partial), and when they are, G1 scans them concurrently. They're less of a problem with ZGC.

As long as virtual threads retain an entrypoint of control flow (e.g. return point from an I/O call), they will also be GC roots. They might not be very deep but they are GC roots.

When a call returns, locals and parameters back up the stack will be expected to be live. Since there's no way in general to create a reference to a stack using JVM instructions (unlike .NET), the stack of every live thread must be a GC root.

No, they are most certainly not GC roots, a fact you can actually observe in a program as described in the JEP (https://openjdk.org/jeps/444#Memory-use-and-interaction-with...).

If you want some more detail, when a virtual thread is in the runnable state, it is reachable from the scheduler (which itself is a Java object, and not a GC root); when it is blocked on a lock or IO, then the lock object or the IO mechanism must retain a reference to it, or there would be no way to unblock it. The thread object has a reference to the stack, which is a heap object (actually, it could be made up of several heap objects).

A thread that is not strongly reachable can provably no longer make progress -- it must be blocked but there's no way to unblock it -- and will be collected even if it has not terminated. It may live forever in our hearts, but not in the heap.

Interesting to read. It's a technical distinction with a primarily implementation difference, which I don't yet understand (i.e. have not taken the time to read yet), but I infer from the fragment that I did read, that there is some degree of semi-magical hoop jumping going on to make the CPU stack live in a Java heap object to which a reference can be taken in Java code.

Objects are obviously rooted for blocked virtual threads that may resume - a formal understanding of them being GC roots - but the implementation appears to be by taking a reference to the heap object containing the stack at the moment of being blocked, presumably by a JVM native method or similar.

> Objects are obviously rooted for blocked virtual threads that may resume

If by "rooted" you mean reachable in the object graph when starting the traversal from the roots, then yes. If a blocked thread isn't reachable, there is no way to call its unpark method that resumes it.

> the heap object containing the stack at the moment of being blocked, presumably by a JVM native method or similar.

Yes, we implemented virtual threads on top of continuations that, in turn, are implemented inside the VM. Their stacks are reified as heap objects.

> Those new fancy threads are stored on heap and need to be garbage collected. If you buy into ads and believe that you can create for free millions of threads, then, well, it is not gonna work on production.

A typical request easily creates 100's of objects that needs to be garbage collected. Adding a single thread object on top of that means absolutely nothing. And how much garbage to you think reactive frameworks create? I can you tell you it is a lot more.

> Before getting too excited I advise to watch Tomasz Nurkiewicz lecture on the subjec

Don't waste your time. He is pretty clueless. I remember him complaining about lack of backpressure and composability. But this comes pretty much out of the box with loom.

How does loom address backpressure?

By scheduling virtual threads in an intelligent order.

The same way synchronous code always has: by blocking. Blocking queues with or without buffers offer sophisticated forms of backpressure (with different levels of "slack"). Backpressure in synchronous code is a problem that was solved long, long ago. It only required special handling in asynchronous code.

Based on that I found this reddit discussion where Ron Pressler (Loom project) commented:

"The posts read like they’re leading to a negative conclusion, but at the end of part 2 it turns out that the current Loom prototype does perform as well as async in their experiments ..."

https://www.reddit.com/r/java/comments/kmn6m3/do_looms_claim...

> Jetty server guys tested this and they were not that happy

This was in 2021. Hopefully things have and will improve in the final version. We have to start somewhere.

> No silver bullet, again.

More like progressive improvement. If you always need things to be a 100% better we'll never have new toys :)

The JVM’s GCs are the state of the art, they can definitely deal with that amount of objects easily, I doubt that would be the bottle neck.

You can even use a million native threads without an issue on Linux. But unless they are mostly idle you probably won't be able to serve a million concurrent clients, wether you use threads or not.

Million thread shouldn't be an issue (million times 2k bytes per thread = 2gb)

Yes, there's no silver bullet. Writing async code from the beginning is a much better approach to reducing memory footprint. But fibers/virtual threads/whatever-you-call-them will help scale many thread-per-client apps w/o having to pay for a rewrite as async code. That is worth a lot because the size of the thread-per-client Java codebases is enormous.

That is very much false.

Async code is the wrong fit for Java, and it complicates everything tremendously. There also is no reason to believe async code should consume any less (or more, for that matter) memory than Loom-style concurrency.

If anything, I would assume that, after a few rounds of optimization, Loom will be significantly better at managing memory than a Java async framework.

> Async code is the wrong fit for Java

Yes, async is a bad fit for Java. That's a problem with Java, not a problem with my statement.

> There also is no reason to believe async code should consume any less (or more, for that matter) memory than Loom-style concurrency.

I'm not familiar with Loom. I was referring to async vs. threaded. Async code does make the programmer make state explicit in objects rather than partially implicit as local variable values on the stack, and this is more compact than smearing part of the state on the stack.

Reply on news.ycombinator.com