It'd be great to know exactly what sort of speed improvements this might bring (or if it's really faster at all).
It _seems_ like IJ could be scripted using the plugin system to do things like:
- Load and apply formatting to a 1 MB source file.
- Index an entire project.
- Run a regex find/replace on a code base.
- Load 50 code files and time how long it takes for each to be ready to display.
IntelliJ has an enormous number of features and people use their own subset of them. Not to mention things like plugins and other options for customization. It's a huge space.
I wholeheartedly agree there will always be very weird setups on very weird source, but if they can make the 80/20 run faster, hopefully a rising tide will lift all boats kind of thing
One example of variation is that the users might be writing code using different languages. The parser and static analysis for a project written in Go will be very different than a project written in Java. All the language-specific stuff is different code that has different performance characteristics.
As an analogy, it's sort of like testing browser performance for loading an "average" web page, when each web site has different performance, depending on how it was written. Load times for Hacker News aren't all that relevant for Gmail or the New York Times.
Yes, you can make a benchmark and optimize it, but you might not even be running the same code that real users are running, due to plugins.