Helidon Níma: A Java microservices framework based on virtual threads
helidon.io
helidon.io
for (int i = 0; i < count; i++) {
resp.add(callRemote(client));
}https://docs.oracle.com/en/java/javase/20/core/virtual-threa...
The idea is that you can map a million of these virtual threads on to a small number of platform threads, and the JVM will schedule the work for you, to achieve maximum throughput without needing to write any complicated code. Virtual threads are for high throughput on concurrent blocking requests.
EDIT: the sequential calls to "callRemote" still process in sequence, blocking on each call. Don't get confused there. But the overall HTTP request itself is running on a virtual thread and does not block other HTTP requests while waiting for the callRemote invocations to complete.
Yeah I got that. But then the use case seems to be much narrower, because not every scenario is one where many concurrent requests come in at the same time. If you write non-blocking code you'll get high throughput no matter the number of requests. Ok, maybe non-blocking code is harder to write (and I don't think it's actually that hard if you use, say, Kotlin coroutines), but honestly this seems to me like something that developers should learn eventually anyway.
Or maybe it's me being weird. But I remember 10 years ago when node.js was being hyped for being "fast" because it was "async" and now suddenly using threads and blocking operations is again all the rage now.
Thread thread = Thread.ofVirtual().start(() -> System.out.println("Hello"));
thread.join();Presumably I could just keep an existing server written in Spring MVC or a similar technology and just wait for the underlying container to support virtual threads to get the same benefits. I believe Jetty already does support them. So why would I need a new framework?
You don't need to switch frameworks
The point this train wreck of an example is trying to make (and failing) is that it's fine to write sequential blocking code in your request handlers. They can take as long as they want ... seconds, minutes, days .... because the handler threads are now an infinite resource as far as the JVM is concerned.
https://inside.java/2021/05/10/networking-io-with-virtual-th...
var remoteResultA = callRemoteA(client);
var remoteResultB = callRemoteB(f(remoteResultA));
Where handling a request means you have to make two http calls and those two calls have some order dependence. I.E. the result of one is used to construct the call to get the other.What virtual threads give is the ability to write the code like I did above and still get ideal performance, as opposed to.
callRemoteA(client)
.then(remoteResultA -> callRemoteB(f(remoteResultA)))
.then(... your world lives here now ...);
Virtual threads are, from a language semantics point of view, the same as regular threads. Virtual threads map maybe 10000 "logical threads" to a handful (maybe 8 or so) actual operating system threads.The big difference is that for each regular Thread you have exactly one backing Operating System thread.
Previously in Java if you wanted to avoid code that blocks the OS thread you would need to use callback based code or reactive programming: https://www.alibabacloud.com/blog/how-java-is-used-for-async...
Javascript moved on from callbacks to async/await, which colors the async functions and requires the programmer to be aware of the issues related to async/await.
For Java Project Loom takes a different approach. For example with server applications, the server would start running the code in a virtual thread and most of the code that looks like blocking will actually work like unblocking code. You get to use regular blocking code but you get async performance.
When the code needs to wait for a response, JVM can unmount the virtual thread from the OS thread and then the JVM can run another virtual thread in the OS thread. Once the response is returned and the OS thread is free, JVM can continue executing the original request.
There are some caveats to virtual threads. There is some overhead (but not a lot), some IO calls still block (but you don't get an indication about using blocking IO) and you might stumble on new kinds of bugs related to resource exhaustion (try reserving a million OS file handles).
But no more async/await, reactive programming, callbacks. Mostly just regular imperative blocking-looking code. I'm very excited about that.
Having fun using async/await in loops? https://zellwk.com/blog/async-await-in-loops/
An actual blocking model is much easier to grok.
It might look like surprsising behavior maybe, but putting the async keyword outside and looping without non-awaited callbacks is the only possible way to keep the semantics sound, no?
In other words, forEach would have to return a Promise too. That's where the function coloring rightfully annoys programmers to catch their mistakes, there are if cause also lint rules for this.
AFAIK there are two active proposals that aim to provide this natively (and additonaly enable working with lazy and/or infinite iterators or generator functions):
AsyncIterator and tc39/proposal-async-iterator-helpers
So, if something like Tomcat "simply" switch its internal threading model from the current model to the virtual model, would the applications know any different? Could you drag and drop a JAX-RS service or a Java Servlet into a container running on virtual threads, or would rapidly run into some leaky wall?
Seems like a virtual thread model of Tomcat would be quite useful in some scenarios.
With Nima, you just write the code to do the thing.
It's a huge difference.
If you have the wisdom to see how toxic annotation-driven programming in Java is, you will like Nima.
Helidon Nima is more comparable to building services in golang. It's more like Vert.x but with a serial API.
Granted, they may not be used much, but if you have an existing application it's probably easier to migrate away from the annotations than to switch to a different framework altogether.
https://www.infoq.com/news/2023/06/structured-concurrency-jd...
At the moment, my apps are using either the JDK HttpServer, which is easy to use, but of questionable robustness and lacking websocket support, or a Netty-based server, which works very well and supports websockets, but is often awkward to program against because it's asynchronous. Nima should combine a nice simple synchronous API with a fairly luxurious feature set and good scalability.
Of course, it will turn out there are things terribly wrong with it, because this is real life. But for now i'm optimistic!
[1] https://intellij-support.jetbrains.com/hc/en-us/community/po...
The other problems I have is that as soon as you need to do anything blocking (e.g talking to SQLite or RocksDB over JNI), the whole thing falls apart in terms of threading. Doing a HTTP request somewhere in a reactive chain that does some blocking operations - well shit now you're blocking the reactor-http-netty-* threads, which are fixed so you're worse off than with thread-per-request at that point. Or even just using Caffeine cache, or any caching library really, but Caffeine is one of few that prevents async stampede - well shit now you're running on ForkJoin pool during cache misses, but "reactive" threads on cache hits. You can see how this quickly becomes a tangled mess of thread-hopping and avoiding blocking event loops, and it's extremely complicated to get this right with how subscribeOn/publishOn works and how the profiler can't tell you which threads runs what code because the stack traces are filled with garbage.
You also opt out of a lot of what Java offers in terms of synchronization. Using the synchronization keyword is a big no-no, the JVM can park the thread while waiting for the lock. Now that thread cannot schedule other tasks just because that one task ran into lock contention, and it's highly likely a fixed pool of threads. You're basically left with CAS, AtomicLong/AtomicBoolean/etc, and that's it. I've even seen BlockHound, project reactor's tool to help you find if you are blocking somewhere, trigger "Blocking call to LockSupport.park()!" during a ConcurrentHashMap lookup. Add to this that most caching libraries are implemented on top of ConcurrentHashMap. Getting async right in reactive code has a huge mental overhead and you need intricate knowledge of the libraries you depend on, the platform you run on and what it is that actually enables async in the kernel, and where it is unavailable. Good example of this; that UUID library you use uses a CSPRNG. Have you set the JVM flag to read it from the non-blocking /dev/urandom rather than the default /dev/random on Linux?
I think reactive programming in Java is decent as long as all you're really doing is accepting/performing HTTP requests and talking to a database that has a reactive client connector. In essence, anything that strictly deals with networking, since that's all that can really be made truly async currently, thanks to epoll/kqueue/iocp on linux/mac/windows respectively. Hopefully io_uring will broaden that to file IO as well eventually, but unfortunately that's linux-only for now.
Contrast this with Go's scheduler that just takes care of this for you - no need to think about it. Or Rust + Tokio, where it is explicit with tokio::spawn_blocking and offering async counterparts to std Mutex, RWLock, Channels, etc that doesn't block the current thread on lock contention.
[1]: https://github.com/ben-manes/caffeine/blob/master/examples/c...
Everyone is free to use Java in vi with make, if they feel happy doing so.
In fact, there were hardly any Java IDEs when the language was released in 1996, which were quickly provided by Smalltalk, Delphi and C++ vendors.
I became an XEmacs user in 1995, because everything else in UNIX just sucked in comparison with PC, Mac OS and Amiga IDEs of the time.
As for messy code navigation, I have similar experiences in large C codebases, so maybe C is also unusable without IDEs.
All my co-workers use VS Code for TypeScript, but I feel crippled when I can't use WebStorm. Not because I can't program without it, but because I'm missing a powerful force multiplier.
I find the whole notion of software engineers rejecting software applications being useful as a concept while it being the primary focus of their own profession quite fascinating.
Personally I use various IDEs, light weight IDE/Editors and even vi/vim depending on the language & situation.
Think of it this way:
If I asked my team to develop a project/POC with this framework and they came back to me saying that they have to wait for (or prefer to wait for, or whatever) jet brains to add java 21 support to do this, I would not be very happy.
But writing any language is much more productive with a good IDE. For me, the added value in using Java 21 now rather than in a few months is not enough to outweigh giving up the use of an IDE.
Time to add a format check to pre-merge CI.
Being a bit entitled regarding support of preview features.
If said developer is savy enough to use a preview SDK, they should be able to cope with the downsides of lacking all the nice support.
It is not mere IDE Dependency. It is IDE Supremacy. Java is the leader of the IDE master race - the other PL IDE's need to squint to see how far ahead Java IDE's are.
I've seen many frameworks shared in hackernews. But rarely do I see anyone commenting on their IDE's capability to be able to use that framework for go-lang, rust, c, c++, etc.
For example, Rust tokio 1.0:
https://news.ycombinator.com/item?id=25520353
I don't see anyone mentioning an IDE there.
In what concerns C++, have a look at VCL, Firemonkey, Qt, Unreal, Godot, DirectX, Metal,...
All Apple frameworks for Objective-C and Swift.
Just like how our current civilization is dependent on industrialized agriculture and distribution systems.
We cannot imagine how it would be possible to live without those functions. In a way, yes, it is "supremacy", but only if you have access.
Pretty sure this works with Java 20 with the loom previewed enabled.
Kotlin is great for fixing some legacy design oopsies (Java really should have been smarter about null and it can't be fixed without breaking legacy code), but I wouldn't expect it to do something that was impossible or it infeasible in Java.
OTOH, I'm unconvinced that virtual threads make coroutines, reactive APIs etc. obsolete. Those are about structured concurrency, and with projet Loom, the structured concurrency proposal is still in the incubating stage.
and reified generics, if you care about that sort of thing.
Plus immutability/final by default, the stdlib being a joy to use, etc.
Being syntax sugar isn't enough to keep a language going.
Long are the days where it was proposed to have parity alongside Java in Java EE, and Spring used to talk about it as it does with Kotlin nowadays.
Or when Grails was a common subject of talk proposals at German JUG meetings.
Maybe Grails is no longer used as much (like Rails itself), but Groovy found other usages since then, like https://spockframework.org/ and Jenkins pipelines (https://www.jenkins.io/doc/book/pipeline/syntax/). It's not going anywhere, and I see no reason for anyone to be upset about it.
I never saw Spock being used outside Grails projects, and Jenkins seems to become less and less relevant in modern cloud pipelines.
I actually think Groovy sits a bit different to other JVM languages in this respect, because it never was at its core, pitched as an alternative to Java (for a brief moment it was considered in that light, but that was mainly because Java itself became so stagnated). They aren't competing to be the same thing the way other JVM languages are - Groovy is trying to be a really good scripting language and Java is trying to be a really good application language. Similar to how Clojure will probably never be popular but it will probably also never go away.
But it's not necessarily a "don't go there" sign that there's a shorter average lifespan. System languages can't really go away unless the whole platform can and are doomed to be a bit clunky. Picking a language from the category that might only have a 15-year heyday is fine for a lot of apps if it's otherwise better than the system language.
> The technology preview is now available with Helidon 4.0.0-ALPHA5. We will continue releasing previews as the development evolves.
and in the default branch `helidon-3.x` there's no `nima` path too, same for a `helidon-4.x` branch.