I would set up some kind of performance testing environment of your core pages. Then I would let the numbers guide your decisions.