So while we can continue to squeeze more onto a smaller die, providing more capabilities on the same processor looks like it will win out for some time yet.
I'm thinking of them all on a chip (or even same core) with external DRAM shared, and cpu cache shared.
> and having them share memory has all the common problems with race conditions and cache invalidation
That's pretty well a solved problem, or modern OS's wouldn't work.
> providing more capabilities on the same processor looks like it will win out for some time yet
I agree, for now, tomorrow's cpu will look like today's. But further out, I expect my view might perhaps start coming to pass.
GPUs seem to on the right path here as the trend toward a thread manager sending data to various CPU/ALU/SIMD/matrix/whatever units.