Hacker News Reader

Comment by chrisseaton | Hacker News Reader

Why do all JITs need to run in <100ms?

If you run your trading app once a day, all day, and warm it up for an hour first, and want max speed, why do you care if it takes a few seconds to compile?

View on HN

If that is the scenario that you care about, then you can use profile guided optimization in statically compiled languages.

Clang has instructions on how to do so here: https://clang.llvm.org/docs/UsersManual.html#profile-guided-...

But that only gives you info from previous runs. A JIT profiles the current run. That's strictly more information than an AOT compiler has.

Yes, that's strictly more information. You still better collect it quickly, as you are compiling your functions each time they run.

Besides, people rally don't like that warming-up period. But it's not the current bottleneck.

Anyway, JIT has a great potential. Just not for desktop software, or network services, or your trading app. It can be great for scientific computing, for example. But that's potential; currently it's not any good on practice.

Well, Android team decided that AOT introduced in Android 5 was a mistake and they went back to a mix of first level interpreter handwritten in Assembly => JIT + PGO => AOT + PGO(taken from JIT) when idle.

Also the large majority of modern Windows software is actually written in .NET, with small C++ pieces.

If you're running code all day long and profiling it dynamically the whole time, then you're going to pay for it. There is overhead to JIT, and there is overhead to profiling. There is no overhead for a statically compiled program, no matter how you profile it.

I still need evidence that my real time trading app running an entire gcc in it makes it faster. I have no problem with JIT in theory, everything you say makes sense. I've never seen this trend in practice though. Either the optimizations you're talking about can be utilized too rarely, or they are too complex that most JITs don't implement, or they're too expensive that it doesn't give any net gain.

Well, it doesn't actually make sense in theory. While JIT theoretically can profile a current run it also adds a JIT related runtime overhead. It needs to waste a lot of resources on both profiling and recompilation to make use of any new information. That's always going to be strictly slower than PGO techniques, with basically hand picked optimizations for each application and zero overhead.

Reply on news.ycombinator.com