There is no reason an IMGUI cannot efficiently construct a hierarchy in the background and do a fast Flutter-like linear layout pass on it before processing input and rendering it. Then throw this away and do it all over again for the next frame (or the next time input arrives, if not used in a game-like context). I've experimented with this, and speed is no impediment if implemented correctly.

IMGUI is about the immediate mode interface, not about layout/rendering/input handling happening at the same time as view functions are called. And don't do "if (Button()) { ... }", but use lambdas for event handling, so the handlers can be invoked at the proper time when everything is laid out.