You're putting the cart before the horse. The reason that performance has not improved in the desktop and gaming are due to hitting fundamental limitations in the design of hardware.
I would argue that the rising production costs to produce games that actually make use of ultra-advanced hardware are more to blame for the slowing of the graphics race.
You can always do things faster by doing things in parallel. It just gets much harder to program them.
But that's beside the point: the games industry drives the graphics card business, but that is only a one part of the consumer computer business. General purpose processors are a larger part, and processor performance improvement is not the exponential curve it was a decade ago.
For a long time, processor architects were able to increase the frequency, increase the cache size, and increase the pipeline depth (to allow for more instructions in flight at the same time) to yield more powerful processors. As Moore predicted, processor architects kept getting more transistors to play with, and they were able to make processors more powerful by making designs that were "like the old one, just more so." But they've hit fundamental limits in the design: increasing the clock speed and the pipeline depth at the same time means that you have to communicate the same amount of data over a longer distance in a shorter amount of time in the silicon. We've hit the point where it's not feasible to do that anymore.
Hence, multicore. Processor architects are still getting more and more transistors to play with, so instead of using them to make a single core more powerful (as they did for a long time), they're using them to make multiple cores. But you no longer get the "for free" performance boosts that you did when you increased single-core performance. Now we need to change how we make software to take advantage of this new hardware. And some software can't take advantage of this new hardware.