IT's version of the Laws of Thermodynamics is that nearly every interesting action you can do programatically is going to take at least O(log n) time, where n is the amount of data you have (at rest, in flight, or both).
If I have 16x as much traffic as I used to deal with, my load isn't 16x higher, it's 64x higher - if I'm fortunate and we've engineered for scale. But it could easily be three orders of magnitude higher (16164) if I'm not.
For the 64x scenario I tell my boss to buy more hardware and I put medium to high priority tickets in the backlog to improve the worst cases. In the 3 orders situation, those tickets bump all of our other priorities, and there are more of them. Performance is now a feature. It sucks up a significant and very visible fraction of our engineering budget, which comes with its own sort of opportunity costs.
If you try to scale vertically, we all know that except at the low end, buying a server that's twice as beefy costs far more than twice as much. If you go horizontal, there are plateaus where your network topology has to get more complex (hardware cost, maintenance cost, speed, pick two). Personally, I think the biggest lie in cloud computing is that 10G ethernet fixes all of your network topology problems (ie, it's treated as magic that you don't have to worry about). As disks and PCIe get faster over the next couple years I think that'll be back on people's radars.