Hacker News Reader

Comment by ori_b | Hacker News Reader

Why can't the jit inline the generated code?

Static dispatch vs. dynamic dispatch.

Assuming the functions and types are known in advance, no virtuals etc: given something like `Main` calling `DoSomething` calling `GetPropertyValue` which is

    public int GetPropertyValue(MyObject myObject) { return myObject.PropertyValue; }

When RyuJit compiles your application, it can see these methods and how they're called. `PropertyValue` can be inlined into a field access, `GetPropertyValue` can be inlined into `DoSomething` which in turn can be inlined into `Main`.

When you do something like the article:

    Func<TestClass, string> getDelegate = 
        (Func<TestClass, string>)Delegate.CreateDelegate(
                 typeof(Func<TestClass, string>), 
                 property.GetGetMethod(nonPublic: true));
    
    [Benchmark]
    public string GetViaDelegate()
    {
        return getDelegate(testClass);
    }

RyuJit can't see that `getDelegate` is the equivalent of `return myObject.PropertyValue`. It's a reference to some function. RyuJit has to compile calling an arbitrary method vs. calling `GetPropertyValue`.

FWIW, the pre-RyuJIT .NET runtime was definitely capable of inlining and devirtualization. I've seen it do the optimization for IDisposable.Dispose calls, off the top of my head. I would expect RyuJIT is capable of doing this in more places.

I don't know if they've ever implemented this for delegates, though.

Yeah my wording is a bit loose. There's a lot of clever stuff that RyuJit etc. do for performance. I believe the JVM has even more clever stuff although I haven't dedicated much time to learning what it does better or differently.

It was actually capable of inlining to the point where it could do some tricks that you'd normally expect only from C++. For example, if you use generics and structs rather than delegates to implement higher-order functions, like so:

    class Program {
        interface IFunc<T1, T2, TResult> {
            TResult Invoke(T1 x1, T2 x2);
        }

        struct AddInt32 : IFunc<int, int, int> {
            public int Invoke(int x, int y) {
                return x + y;
            }
        }

        static T FoldLeft<T, F>(T[] xs, F f) where F : IFunc<T, T, T> {
            var res = xs[0];
            for (int i = 1; i < xs.Length; ++i) {
                res = f.Invoke(res, xs[i]);
            }
            return res;
        }

        static void Main() {
            Console.ReadKey();
            int[] xs = { 1, 2, 3, 5, 8 };
            int res = FoldLeft(xs, new AddInt32());
            Console.WriteLine(res);
        }
    }

I compiled and ran it with 3.5 SP1 x86 (the old 64-bit JIT wasn't good, and won't inline in this case). It didn't inline FoldLeft, but it did inline AddInt32 into the loop - this is from VS debugger disassembly:

            for (int i = 1; i < xs.Length; ++i) {
    017B0106  mov         edx,1  
    017B010B  mov         edi,dword ptr [ecx+4]  
    017B010E  cmp         edi,1  
    017B0111  jle         017B011E  
                res = f.Invoke(res, xs[i]);
    017B0113  mov         eax,dword ptr [ecx+edx*4+8]  
    017B0117  add         esi,eax  
            for (int i = 1; i < xs.Length; ++i) {
    017B0119  inc         edx  
    017B011A  cmp         edi,edx  
    017B011C  jg          017B0113  
            }

Generic code working with reference types can be shared, but passing structs for generic parameters forces JIT compiler to generate separate instances of generic methods or classes for each combination of structs. From there inlining becomes trivial. But yes, being able to pull stuff like this is one of the cooler parts of .NET.

Yes, I understand that. I've worked on JITs and fancier compilers in the (long) past, and I'm aware of inlining and devirtualization.

My question is what is special about the code that is written into memory by code generation that distinguishes it by code written into memory at program startup. I can't think of anything off the top of my head that would make inlining dynamically generated code impossible, short of possibly not wanting to toss away already jitted code

Reply on news.ycombinator.com