New JIT optimizer in the Zing JVM | Hacker News Reader

New JIT optimizer in the Zing JVM

stuff-gil-says.blogspot.com

99 pointsdmit28 comments

I am wary of picking an argument with Gil Tene, who certainly knows his stuff, but I wonder about the AVX2 example.

The post makes it sound like Intel only contributes patches for new instruction sets to LLVM. But they are also active HotSpot contributors.

For instance, here's a post from an Intel employee contributing optimisations to use AVX512:

http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/...

I have no idea if HotSpot optimises the specific loop Gil posted, but I've seen Intel contribute a lot of upgrades to use AVX2 and AVX512, like optimised compression intrinsics. And as noted, LLVM is not really designed for Java and they had to spend three years putting in optimisations that Java-specific JITCs have had for a long time. How many contributions to LLVM will be directly applicable to Java, given that all the other contributors are focused on C++-style languages? Use of new instructions, sure, in some cases, but they aren't going to contribute intrinsics for the Java APIs.

ReadyNow and C4 are very cool. Java has needed these for a long time and Zing users have been benefiting for a long time. However, Java 9 gets AOT compilation for some cases, which gives at least some of the benefit of ReadyNow (there is still warmup time, as HotSpot AOT does not cache profiles and re-profiles each time, but you get a lot closer to peak performance a lot faster). And Shenandoah is driving down pause times for huge heaps, which is where C4 shines.

That said, Java 9 has not yet shipped, and Zing has. So congrats to Gil Tene and his team. The competition between Zing and HotSpot only helps.

> I have no idea if HotSpot optimises the specific loop Gil posted,

I'm actually curious to see what the output assembly of LLVM is for that loop, because JDK8 does not optimize a branch in a loop.

But the JVM does do a pretty decent job using ymm<n> registers if you write the loops without branches.

But you end up with really obscure java code like

"(((a - b) ^ (b - a)) >>> 63) ^ 1"

To convert a branch into an integer.

you certainly can vectorize branch in a loop. Unroll loop, compute conditions in parallel, use result as a mask for conditional computation.

The parent's point was that the JVM didn't vectorize, not that it wasn't possible.

I had actually posted an annotated assembly output for one of this example vector tests (the doAddArraysIfEven one) here: https://groups.google.com/d/msg/mechanical-sympathy/EMIBqjX4...

Enjoy!

Pretty sure that code gives the wrong answer for a = 0, b = 1 << 63 (assuming the goal is equality represented as 0 or 1).

If you're testing with JDK8 you're missing the last two years of work on the optimiser. I'd be interested to see what Java 9 does, as it's (supposedly) coming out in a few months.

JDK9 has the capability to vectorize a loop with branch inside. I tried JDK9 early release with a simple testing case myself.

I'm in no way arguing that Intel and others do not also contribute to HotSpot. It's just that MUCH more attention and seems to be paid to LLVM, and the amount and impact of contributions there are bigger.

As to the AVX2 work contributed: there are all sorts of "easier to integrate" AVX2 uses that had been added to HotSpot over time (especially with intrinsics, which tend to be nice and isolated from the rest of the code gen), but the vectorizer in C2 is not nearly as capable as the one in LLVM, and the results speak for themselves. And I expect o see a similar gap open when AVX512 becomes available on Skylake EPs soon.

On the "how many contributions directly applicable to Java vs. C++ question": we were surprised at how readily those "C++ focused things" apply to idiomatic java code. The LLVM contributions can probably be better describes as "expecting certain patterns and idioms that are common in C/C++". And once you bring the Java code to LLVM IR form and "remove the breaks" as much as possible (e.g. fix how things like safepoints, GC barriers, and depot points used to defeat most optimizations), the patterns seem to be there in the Java code as well. The vectorization u-bench thing here is certainly a good example of that in play. We didn't massage it to find the vectorization opportunity. I didn't even get specific directions or tips on "write it this way". I just wrote some "I think this loop should be vectorizeable with AVX2" pieces of java code and the existing vectorizer seemed to have just picked them up and had its way with them.

Thanks for the response!

Yes, I can well believe that LLVM has put a lot more effort into auto-vectorisation than HotSpot has, given the relatively different ways in which Java and C++ tend to be used. And I don't see much effort put into auto-vectorisation in Graal either, from my occasional browsings of the commit logs. The research in HotSpot JIT compilers seems to be focused more on compiling more dynamic languages like JavaScript and Ruby faster, rather than C++ style numeric code: I guess reflecting a focus on business apps that probably don't contain many hot array loops relative to scripting language use.

I look forward to benchmarks. The blog post contains a graph that purports to compare HotSpot vs Zing but there is no Y axis, so I guess it's meant for illustration of the basic idea than actual performance comparisons.

While optimization for dynamic languages is a cool research area for a JIT compiler, and may one day help expand the use of dynamic languages on the JVM, the VAST majority of code that runs on JVMs in written in Java, Scala, and Clojure, none of which is dynamic. What you do see in those languages is plenty of idiomatic loops, stream, and buffer processing. All of which benefit greatly from the new vectorization capabilities in AVX2 (and the soon to be widely available AVX512).

perhaps someone should advertise AVX2 and AVX512 under the tag "It's not you father's numeric-processing-focused vectorization"

As for the charts in the blog post: yes, it's a depiction. The Y axis gains will vary by application, obviously. And we've seen that gain range form a few % to high 10s of %s with real workloads, measured by real customers who use them. Use cases span a wide range, including databases, in-memory query engines, search and indexing, protocol gateways, stream processing, messaging, analytics, and matching engines. None are things that classically make you think "numeric processing", BTW.

Yes, I'd very much appreciate some sort of blog post or link on how AVX2/512 can accelerate ordinary business type apps. I know it can be used to accelerate things like hashing, checksumming and so on, but if it's widely applicable to the sort of code normally found on the JVM, that could make a very interesting set of articles.

Thanks for the followup!

> some day someone will eventually find (and ship?) some other holy graal

Was that the dis I think it was?

I think it is, sir.

I'd love the instant start-up time for Java-based tooling like IDEs/builds/etc. (assuming it's as slick as they make it sound).

It would be nice if there was a developer version, e.g. not for commercial server usage, but as your local/desktop JVM installation.

For that purpose I think Java 9 AOT might be sufficient. Let's see if and when Intellij/Eclipse move to that.

I think in the current form that's unlikely. It's Linux only and needs to be built on the exact same CPU. This essentially means that you have to compile on the same system as you execute.

How much it cost to run that Zing JVM per server?

To quote the FAQ:

>The single license annual subscription price for Zing is $3500 USD per server, with significantly lower prices for higher volumes and longer-term subscriptions.

Isn't we can just spin one more VM for each instance instead of paying 3k$?

Not only that but you can also use the money to buy faster CPU.

From where? They aren't making Alphas anymore, or GaN 5GHz LN2 cooled monsters. At this point SiC is the material as power dissapation has become the limiting factor.

Faster CPU does not eliminate GC pauses if your workload consumes a large amount of RAM. This is one of the other benefits which Zing provides.

You can load balance your app for 3.5k $ on additional server. By reducing load, time spend in GC will be less. Additionality by having more hardware you can use more aggressive strategy[0].

[0]http://www.cakesolutions.net/teamblogs/low-pause-gc-on-the-j...

I have taken Zing for a spin on 24 c3.8xl ec2 instances running Elasticsearch with 32G heaps and just under 2 billion indexed documents with millions of updates/day. The results are considerably better. If I just went up to 30 or even 50 ec2 instances on HotSpot/Openjdk, I would see no performance benefit with regard to GC. And I have performed this test. I'm not saying dont use Java because of GC, but I am saying don't overestimate the value of more efficient garbage collection in any case.

Shoutout to the Azul guys, Simon, Elaine and Paul for coming on-site and watch me compile all their extensions on Amazon Linux and help me perform all the testing.

For actual end-user experience and impact of using Zing on AWS/Cassandra, check out: https://twitter.com/garryturk/status/859883566267871232

VMs adds a noticeable overhead / performance hit.

Besides the benefits the new JIT provides, Zing will help you if your workload runs with / consumes over 8GB of RAM. At this point, HotSpot GC pauses [1] starts becoming noticeable (i.e. jitter).

Notes:

[1] - http://www.azul.com/resources/azul-technology/zing-consisten...

They have an AWS offering also https://aws.amazon.com/marketplace/seller-profile?id=5829a41... . I tried it a year or so ago and my Clojure app "just worked". (I am not affiliated with Zing/Azul)

Reply on news.ycombinator.com