Reminds me of the PS/2 CELL Architecture, with the SPUs having limited local memory, and needing to DMA stuff around in a streaming fashion to build up larger programs.
Nothing really prevents an architecture where part of the address space is core-local (with low latency) and not directly accessible from other cores, and the rest points to a (much higher latency) shared memory pool. I have a feeling this would be much nicer to program than a Cell or the average GPU.
Many GPU's have this kind of local scratchpad memory. Nvidia GPU's even allows the programmer to partition the local SRAM between the scratchpad memory (called shared memory in CUDA docs) and L1 cache.
Minimizing ownership transfer cost across hardware elements seems like a fundamental concept to me that future architectures will need to specifically optimize in order to maximize the benefits that can be reaped from "task specific" hardware.
The more cores we add the more we have to gain by allowing them to be different from each other I think ...