Comment by chrisseaton | Hacker News Reader

Comment by chrisseaton | Hacker News Reader

I'm anticipating a lot of confusion and people talking past each other in this thread.

C2 already does escape analysis, and scalar-replacement-of-aggregates to turn objects that don't escape a compilation unit into dataflow edges, which if possible is far better than the stack-allocation in this article because you'd rather not write out to the stack if you don't have to, and you'd like unused parts of the object to disappear. Graal then adds partial-escape-analysis to float reification of an object to the last-safe-moment.

What this does on top of that is then to cover the corner case where an object would be safe for SRA, except something requires the object to have the same layout as on the heap. I think a concrete example of this is some intrinsics and merges where an object needs to be heap-allocated on just one branch, so that code following the merge can have a full object in both cases and not care that one has been allocated on the stack.

The key to avoiding confusion - 'allocating on the stack' in this case means literally allocating the full reified object in stack memory, whereas in C2 at the moment when people say 'stack allocation' informally they mean SRA.

C2 already removes reference type allocations when they don't escape a compilation unit, which is something I think C# doesn't yet do at all for some reason I don't really understand.

> C2 already does escape analysis, and scalar-replacement-of-aggregates to turn objects that don't escape a compilation unit into dataflow edges, which if possible is far better than the stack-allocation in this article because you'd rather not write out to the stack if you don't have to, and you'd like unused parts of the object to disappear.

Yes, and I'd also add that SROA typically enables lots of optimizations since it lets you treat object fields as though they were ordinary local variables.

It appears that the .NET Core CLR does perform some rudimentary escape analysis: https://github.com/dotnet/coreclr/pull/6653

Right - and that supports the stack-allocation, not SRA, which is what I think C# doesn't have - possibly because they prefer explicit value types over trying to do it automatically through SRA.

> What this does on top of that is then to cover the corner case where an object would be safe for SRA, except something requires the object to have the same layout as on the heap.

I'm a little surprised that see such noticeable performance improvements. Obviously benchmarks etc etc, but I didn't expect these cases to be quite so common.

Sure, it surprising as an outsider. But nor is this brand new.

Recentish versions of .NET (Core) have introduced ValueTuple and ValueTasks. These types avoid the heap allocations with Tuple and Task.

Hopefully Microsoft think this makes enough real world difference to justify the pain of two sets of types.

Those are structs, though. This work is about providing similar performance advantages for objects (in certain conditions).

Which begs the question: if the compiler can automatically transform objects anyway, why do we even need value types/structs?

Or asked differently: what is the difference between an object with properly overridden hashCode() / equals() methods and which is effectively only being used as a data container, and a struct?

If you pass an object into a function, the code has to work with any subclass. This means the data has to be behind a pointer since the size is unknown and method calls have to go through a vtable. If you can see the full lifetime of an object you can specialize away until no pointers are left but in the general case this doesn't work, or makes performance worse because you unbox and rebox repeatedly whenever you call methods.

For final classes unpacking into registers might work, but at least in the jvm you can subclass final classes via reflection. If I understand correctly this avoids repacking when calling methods that expect the normal memory layout by actually allocating the the normal object layout - just on the heap.

Makes sense, thanks for the explanation.

So some JITs can be given hints that certain objects are just values, and things like identity will never be used, and thus they can be boxed and unboxed as needed, but this is a pretty low level mechanism, and it’s probably not the thing to expose to most users as it can be brittle both in terms of safety and performance. There are a few other internal annotations in the JDK which are useful in the same way but aren’t exposed for similar reasons.

Adding value types to a VM or language means exposing that sort of feature in a way that can be used by programmers and provides the sort of safety guarantees they are used to.

Those transformations are not allowed in every situation. Usually they are only possible because of aggressive inlining that eliminates function borders. Once you cross multiple functions after optimization escape analysis is unlikely to work out. The best thing you could perhaps do is add optional restrictions that make escape analysis easier. The end result would be something very similar to the borrow checker in Rust. However, it would be much easier to not rely on escape analysis and just add value type semantics directly into the language.

Because escape analysis isn't as powerful as it might seem. Indirect calls in particular can cause objects to be marked as possibly escaping, which matters a lot in languages like Java where most calls are virtual.

> languages like Java where most calls are virtual

Most Java call sites are in practice monomorphic. They may look virtual according the language spec, but that isn't how they're really implemented.

> if the compiler can automatically transform objects anyway, why do we even need value types/structs?

It cannot always without affecting the semantics. An array of structs will still have to be an array of references in most cases, for example. The compiler isn't sufficiently smart to figure out such things beyond trivial examples.

In General it applies to any optimization: the compiler can figure out a subset of optimizations the programmer can. That's why not giving the control of memory layout to the programmer is objectively bad idea.

Also, when you need performance, not relying on compiler to figure out the optimizations is easier than ensuring the optimizations happen. Compilers are quite unpredictable.

Reply on news.ycombinator.com