I don't think JITs are the only way to address that trade-off in compilation time vs runtime performance metrics, and whether their benefits are worth their other costs is an interesting, program-dependant question, but they have strictly more information available than a naive compiler and can potentially use that to make better decisions.
Here's a reason why this might not be a super great thing to do, if you're thinking of it from a language design standpoint:
- Profiling and rejitting costs you memory and makes all code pay a tax. The biggest tax is the safepoint/OSR tax. It costs significant memory and some small amount of time (maybe the time cost you add from comprehensive OSR support is like 1%-5% overall, but I'm not sure, because it's hard to isolate this cost and measure it). So you don't want to build a system that supports JITing unless it's going to give you big wins. That implies languages with lots of dynamic typing.
- The languages that most benefit from JITs (the win they get consistently overcomes the overhead of JITing) are the ones that have so much dynamic typing features that compiling them to native is a super hard problem.
Java is one of the few languages that is both dynamic enough to benefit from JITs but static enough to be possible to compile to native. And even for Java the native compilation is a lot of effort to get right.
Note that "possible to compile to native" to me means: compiling to native produces something that performs well so there exists some benefit to actually doing it. Like, I would expect compiling JavaScript to native to produce something that doesn't perform well at all.
So, basically, the issue is that most languages are either in the "benefit of JIT is smaller than cost of JIT" category because they have adequately static typing or in the "must have JIT and cannot compile statically" category because they don't have static typing. Not a lot of languages are in the sweet spot where doing both would help, but such languages exist (Java) and there's no reason why they can't do what you suggest and they may already do it (seems like a natural thing for Graal to do).
Every user should have it's own PGO and regularly refresh it to be competitive with the advantage of JIT profiling
A compiled language with a "micro-JIT" (e.g. for v-tables) seems like an interesting idea to me.
- gcc - clang - msvc - intel
In some sense we can always fully specialize a program to its current input and make a lot of radical changes to data layout and represesentation. But in languages like C/C++ where programs can "see" struct/array layout, member order, pointers as integer values that have fixed order, spacing, grouping based on the program specified data types, this is prohibitively complicated.