There are many architectural differences between CPUs and GPUs that make it difficult to draw the kinds of conclusions suggested by these graphs.

Data parallel algorithms with high computation-to-communication ratios, good data locality and low local branch divergence will see these kinds of speedups when rewritten for GPUs. This is an important subset of algorithms, but don't expect GPUs to replace CPUs for more general-purpose computation any time soon.