Using "ondemand" rather than "performance" can do quite a bit for your interactive performance as well. With "ondemand" your CPU runs at half speed until you've stressed it for a bit. So if say, switching to another desktop takes 0.1 cpu seconds, but the threshold is that the CPU must spend 0.5 seconds working before it kicks into full performance then all those UI actions might only execute at half speed.

So personally, I have my desktop CPU (i7) running at performance setting. That was clearly noticeable for me for short bursts of activity. It's possible you could settle on doing that for just some of the cores (I don't know if that makes sense, since they're in a single package).