In my experience, a lot of performance problems boil down to really stupid problems, like simple code using the wrong data structure out of convenience (e.g. linked lists instead of arrays for lots of randomly-accessed data), or structured in a bad way (e.g. allocating a lot of small pieces of memory all the time). Often times, there are cheap performance wins to be had if you occasionally run the product through a profiler and spend couple of hours fixing the most pressing issue that shows up. Couple of hours isn't much; there's enough slack in the development process to find those hours every month or two, without slowing down your regular work.