For interpreters, that's highly likely to be true, especially if they are mainly spending time in switch tables or executing basic opcodes.
In the past, at a time when I worked on a very performance sensitive codebase that was also limited in scope, we compiled with -Osize and did all the loop optimizations we wanted manually (and with pragmas). That produced faster code than -O2 or -O3.
A more granular control over optimisation would be good, however.
In other words, loop unrolling is, more often than not, harmful.