Know Thy Java Object Memory Layout | Hacker News Reader

Know Thy Java Object Memory Layout

psy-lob-saw.blogspot.sg

66 pointsjavinpaul20 comments

Does this ordering happen by the JVM at runtime, or is it part of compilation? Is there a static analysis heuristic which could detect false sharing? Maybe his could be better solved in the compiler than with implementation-specific hacks like these?

It cannot be done as part of compilation. For example, the superclass you run against may not be the same as the superclass you compile against.

Thank you! You just saved me another day trying to track down why some "supposed" hot fields were causing cache misses.

At some point it would be great if Java added some support for memory "hints" that would allow the JVM to interpret these hacks for what they are.

I agree.

Maybe not news to you, but I think that there are a few things on the cards for dealing with memory layout. I know there is a strong push from certain corners, including Doug Lea, for (object) headerless struct values to be included in Java 9.

There is a hint of a an @Contended annotation to get the JVM to put a variable on its own cache line.

But it seems to me that a very effective fix would simply be to ask all JVMs to stop fiddling the variable ordering.

I'd much prefer a solution that allowed the code to be read easily by humans (ie, wouldn't need to know variable ordering rules), and the hints be there to handle the machine implementation.

That said, I'd take standardizing the variable ordering.

I think that the proposed solutions, @Contended and perhaps some sort of @Respect-My-Variable-Order, is adding more knowledge overhead than a simple rule, the JVM will lay out your variables in the order you declare them and it won't drop any even if it knows you aren't using them.

This feels like the simplest option to implement on both sides, the JVMs do less work and the programmers have simple rules to do in-object memory layout if we like that sort of thing. More than anything else it means we don't have to wait on a new JSR for each type of memory layout we want to achieve.

(I suppose there might be some CPU arch portability that the JVM might be able to do if we go with the annotations - but I think that we may be chasing a fools dream down that road)

> there are a few things on the cards for dealing with memory layout

LOL. Cards... ha. You must be referring to the G1 GC and the memory 'card' design? Ha.

>But it seems to me that a very effective fix would simply be to ask all JVMs to stop fiddling the variable ordering.

That would make the common case much worse. 99% of Java programmers want the JVM to optimize their field ordering for them, to the best of its abilities, so that they can put them in the order that makes sense in the source code and simply not think about whether a different order would be better. It's the kind of low-level implementation detail that Java is all about ignoring; if you want C++, you know where to find it.

I support an annotation-hinting approach; that would keep the common case the way it is today, while allowing people who really want to fiddle with these things to do so.

I would expect that any advantage the JVM gains from variable order fiddling (and variable dropping) is minimal. Whereas the problems we get on multicore systems with false sharing, and other cache-line maladies, are significant.

This is, however, just what I expect to be the case. It may be that in practice the JVM is getting good results from fiddling with our variables.

You may be right that annotations would suit Java better than allowing programmers to layout objects as they wish. The strong motivation for this is that the fix would be simple (and perhaps have no serious downside?) and mean that work can be done without waiting for a JSR to reach maturity.

From what I read the state of the art around this is to use relatively unstable hacks which require very strong knowledge of JVM implementations - see the example code from Martin Thomson in the linked article.

You could still happily write your variables for humans in most cases. I would love to know how large the downside is to fixed variable ordering.

I think instead a @ReorderingBarrier annotation might be a good solution. Like a memory barrier which doesn't allow reordering of memory operations across that point, a reordering barrier wouldn't allow fields declared before that point to be placed in memory after the field it's declared on or any declared afterwards.

It could also be used to control the alignment of the following field, to a cache line for example:

    public class TestClass {
        int a1;
        long a2;
        OtherClassA a3;

        @ReorderingBarrier(alignment=CACHE_LINE)

        OtherClassB b1;
        int b2;
        int b3;
        long b4;

        // ...
     }

If I understand the reordering rules correctly, without the annotation it would be rearranged as: a2, b4, a1, b2, b3, a3, b1.

With the barrier it would be rearranged as: a2, a1, a3, (cache line alignment break), b4, b2, b3, b1.

If they provide an alignment argument then the annotation based approach becomes much more attractive. You could even go all cross platform and have 'alignment=cacheline'

That would be a lot of fun to play with.

Hah, just as you were writing that I had the same thought and did a ninja edit where I switched from saying a specific alignment to CACHE_LINE. :)

> I know there is a strong push from certain corners, including Doug Lea, for (object) headerless struct values to be included in Java 9.

Thank christ. Being able to avoid the per-object overhead when necessary would be extremely welcome.

What kinds of problems are you working on where this change in behavior of the JVM has implications for your software stack? I myself work on a financial trading platform and these sorts of things matter to us. Curious what other kinds of problems/domains are affected by this issue.

Financial trading as well, but lots of real time systems would have similar issues.

Afraid my interest in this comes from building a matching engine. There are definitely other domains where this stuff is important but I bet the financial sector is the biggest employer.

NB: I write a matching engine in my spare time. For pay I don't touch anything as interesting this.

https://github.com/fmstephe/matching_engine

The packed objects proposal is pretty interesting in this regard: http://duimovich.blogspot.nl/2012/11/packed-objects-in-java....

Isn't there a decent amount of overhead in declaring and extending an abstract class? Does the cache grouping outweigh the overhead?

A) It depends on the JVM and B) on HotSpot at least, I believe that it is smart enough to figure out that all virtual methods are of the same type and optimize them as if they were non virtual.

I can confirm that HotSpot is very aggressive at collapsing type hierachies and de-virtualising method calls.

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.117...

This is one of the places where Java can outperform C++. Where c++ is unable to statically devirtualise method calls and HotSpot will just do it.

Reply on news.ycombinator.com