Hacker News Reader

> Some of those systems come from vendors who have very competent teams and invest heavily in their systems, and some have tried fancy, expensive JVMs with supposedly better real-time performance and have failed to get them to work.

This is very important. There's far too much No True Scotsman in typical discussions around this. The fact remains that really good teams try and fail to meet needed performance criteria, even though their app/service meets all other requirements, is well architected, well coded, and the team has experience in GC tuning, pitfalls, etc.

> Maybe it can be done, but that doesn't mean that you're going to be able to do it, especially if you don't design all parts of your application from the beginning to operate within the constraints of real-time GC implementations.

Because... unless your solution is one where you can simply avoid doing a few stupid things that make the GC unhappy, you have to wade deep into managing memory, except you can't do it directly. You have to understand/infer how seemingly innocent code impacts GC, and then suggest/cajole/plead with the GC to do what you want. It required the kind of intimate knowledge that GC is supposed to save you from, but doesn't have the direct manipulation tools.

There are large classes of problems where GC is perfectly fine. Great even. But there are other problem domains where sluggishness or pauses are not acceptable, and not just realtime or soft realtime.

> But there are other problem domains where sluggishness or pauses are not acceptable, and not just realtime or soft realtime.

Azul sells a pauseless collector. AIUI the downside is that you need enough memory headroom and spare CPU cores for the collector to keep up with the mutator threads, which presumably is a significant overhead on large heaps.

> Azul sells a pauseless collector. AIUI the downside is that you need enough memory headroom and spare CPU cores for the collector to keep up with the mutator threads, which presumably is a significant overhead on large heaps.

I think that one of these vendors tried Azul and couldn't get it to work. (I don't know the details. If I understood correctly, the problem wasn't the GC so much as that Azul couldn't run their code correctly or would have required extensive modifications or something along those lines. In any event, they decided not to use Azul's product.)

I agree. In fact I don't think one can succeed in mitigating GC latency unless you understand how very important it can be. The excellent presentation "How NOT to Measure Latency" explains how with the wrong measure latency doesn't seem that important.

The JVM has a lot of other reasons to pause [1]. A big problem with using the JVM as an example is that it's an enormously complex architecture that wasn't really designed for the kind of workloads it is subjected to these days. For example, the complexities involved in having a JIT compiler in a multi-threaded environment that wasn't designed from the ground up for that purpose can be pretty scary (e.g. the method deoptimization issue in the video). The effort that has gone into the various JVM implementations both impresses and scares me.

[1] https://www.youtube.com/watch?v=Y39kllzX1P8

In what sense was the JVM not designed to be a JIT compiler in a multi-threaded environment?

Like (somewhat ironically) Javascript, Java's origins can be very easy to forget, and also like Javascript, both are surprisingly humble. Javascript was banged together in a week, and Java was intended to be a language for developing constrained set-top box UIs in the 1990s. While that does not render them intrinsically unsuitable after decades of development for the sort of tasks that they are put to today, the early days of a language still inevitably color it forever when backwards compatibility is a big concern.

I don't buy this at all. The original Java whitepaper from 1995 mentions both multithreading and JIT compilation: http://www.cs.dartmouth.edu/~mckeeman/cs118/references/Origi...

It even says that the byte-code format was specifically designed with JIT compilation in mind:

> The byte code can be translated on the fly (at runtime) into machine code for the particular CPU the application is running on. For those used to the normal design of a compiler and dynamic loader, this is somewhat like putting the final machine code generator in the dynamic loader.

> The byte code format was designed with this in mind, so the actual process of generating machine code is generally simple. Reasonably good code is produced: it does automatic register allocation and the compiler does some optimization when it produces the bytecode.

The idea that JIT compilation was retroactively bolted onto the Java ecosystem is absurd. Perhaps more than any other technology, Java brought JIT compilation into the mainstream, starting more than 20 years ago.

To distill your contribution to its essence:

Parent: what's wrong with the JVM as a multithreaded-JIT?

You: Java is similar to Javascript. Javascript was terribly designed in the 90's. Therefore Java sucks . QED.

You should try your hand at politics! Don't answer the question (Java), instead, segue into the more comfortable topic/what you want to talk about (Javascript)

Um, jerf gave a reason why Java's history affects it here and noted how it's similar to Javascript in that regard on the side; not the other way around. The JS comparison was not part of their argument.

But that has nothing to do with the question he is responding to; regarding Java's suitability as a multithreaded JIT

Tu quoque.

The JIT and threading / memory model were bolted on afterwards. The JIT isn't even part of the JVM spec. (Unless this has changed.)

The Sun JVM has had a JIT since 1998: https://en.wikipedia.org/wiki/Java_version_history#J2SE_1.2

Which was after JDK 1.0. The JVM was originally implemented as a bytecode interpreter; Sun licensed the Symantec JIT compiler later for 1.1, then eventually added HotSpot several years after 1.0, derived from Smalltalk technology.

The original design of the JVM was for embedded systems, not for running server applications with heaps in the 10s of GB of memory.

The threading model, which allows for near arbitrary sharing of memory, is still a significant source of complexity for both the JIT and the GC and has changed multiple times since the JVM's inception.

Modern JVM implementations have little to do with the original design, but still have to deal with design decisions made 20 years ago for completely different architectures and with only limited foresight.

You were asking about design, not implementation.

The way Java works is there's a language spec and a VM spec and then organizations are free to implement them as they see fit. So JVM design is referring to what's in the spec.

What's wrong with Sun's implementation? Well, they did the best they could, given that there's no design / spec to work from (for the JIT). The memory model was completely replaced in the spec in ~2005 (to work better on multi-core CPUs and stuff), and the threading model was changed in the spec sometime in the 90s (from green threads to native threads).

Most other safepoints pauses are much shorter and far less common than GC pauses on multi-GB heaps.

For some applications those still are a problem, yes, but others would be happy if those were the only ones they'd experience.

Are you familiar with the D programming language? They think a lot about these issues.

By default D uses a GC for classes (allocated on the heap, Java-style). You can disable/avoid the GC with some additional effort and the language tries to help. For example, there is a @nogc annotation, which emits an error for anything which could trigger the GC, although it does not prevent a stop-the-world event from another thread.

Rust is kind the inverse approach. Currently, there is no GC and real time is (relatively) easy. Now they try to introduce an optional GC for convenience.

Mixing GC with (real time) non-GC code is quite desirable, but I have not seen an elegant approach, yet.

It's not exactly for convenience. The Rust GC support isn't intended to be used in general situations. It's for integrating with languages/applications that have a GC, like spidermonkey or V8.

https://github.com/manishearth/rust-gc actually lets you cleanly mix GC and non-GC heap objects (even put pointers to GC heap objects on the malloc heap and vice versa). So does Servo's current bindings to spidermonkey.

  > Now they try to introduce an optional GC for convenience.

Most of the GC work in Rust, at least at this time, is more about integrating with external GCs. Think "Servo uses javascript objects that Spidermonkey controls" or "A Ruby gem in Rust lets the Ruby GC deal with Ruby objects."

It might be easy, after all, to reduce or eliminate those pauses. Try running the Java programs using the G1 collector [1].

[1]http://www.infoq.com/articles/G1-One-Garbage-Collector-To-Ru...

This is probably not a very productive conversation to have, but that simply doesn't match my experience - and frankly a system with "all kinds of performance problems" sounds like incompetence. Short-lived garbage does not cause pauses (GC in a modern JVM can be concurrent, it's only fragmentation that causes pauses), and long-lived garbage generally doesn't need to exist (that's workload-dependent - but the kind of workload where it's important is the kind of workload that's impossible to write without GC). Holding onto objects for longer than they need to live certainly happens, but that's the same problem you have in a non-GC language, and if anything it's easier to diagnose on the JVM which has some excellent tooling around monitoring these things.

and frankly a system with "all kinds of performance problems" sounds like incompetence

long-lived garbage generally doesn't need to exist

Wonderful. It's not the GC it's you who's incompetent. Or maybe you're competent, but your app simply doesn't really need to exist. A more complete shit-HN-says comment I couldn't find if I went searching for it.

So without knowing any details of the Java programs you interact with you're going to make a blind claim that the performance issues are the result of GC pauses?

No. We've spoken at length with the teams that develop these applications.

There's a whole additional class of problems involving the JIT needing to warm up, too, but that's off topic for an article specifically about GC. (Go, for example, would not have the latter problem at all, but the JIT and GC situations for Java are a bit similar. JIT warmup issues can be avoided using AOT compilation, but that has its own set of issues. What seems to be completely missing, or at least was missing last time I looked, was the ability to generate a JIT profile, save it, restart, and restore the old profile.)