I think register files, shared memory and even texture memory patterns are basically the GPU's way of having fully programmable caches, as opposed to cache being a grey box like on CPUs. Overall I think for HPC code it works out better. Looking at vectorized and cache-optimized CPU code gives me the shivers...

Mostly due to the above I think Intel's years long push to try to have people just run their CPU-optimized OpenMP codes on their Xeon Phis was the downfall of that architecture - it just can't win against a data-parallel-first framework like CUDA, mainly because there the data parallel compute thread can be programmed just as such, rather than trying to get the compiler to behave exactly as you want (and in the end giving up and dropping down to assembly-like SSE instructions).