Amdahl's law is common sense, but also the one of the reasons premature optimization is the root of all evil.

A small speedup on the longest (by time) part of the program is usually better than infinite speedup on a short part of the program.