A IC cache miss costs 50x more on average. If your insn and data are small, the overhead of doing more work is less than the overhead of cache misses

If the whole program can fit in the L1/L2 cache, it can speed up things quite a lot!