More on Apple’s A9X SoC: 147mm2TSMC, 12 GPU Cores, No L3 Cache
anandtech.com
anandtech.com
"One explanation may be that Apple deemed the L3 cache no longer necessary with the A9X’s 128-bit LPDDR4 memory bus; that 51.2GB/sec of bandwidth meant that they no longer needed the cache to avoid GPU stalls."
And:
"Our own Andrei Frumusanu suspects that it may be a power matter, and that Apple was using the L3 cache to save on power-expensive memory operations on the A9. With A9X however, it’s a tablet SoC that doesn’t face the same power restrictions, and as a result doesn’t need a power-saving cache. This would be coupled with the fact that with double the GPU cores, there would be a lot more pressure on just a 4MB cache versus the pressure created by A9, which in turn may drive the need for a larger cache and ultimately an even larger die size."
So, basically, while the A9's 4 MiB L3 cache wouldn't have been much to add to the A9X, it wouldn't have helped much and a larger, more costly L3 cache would have been needed to help the A9X performance-wise. As the article points out, this is all speculation.
There’s a huge latency gap between L2 cache and DRAM.
Maybe not too many workloads need data chunks that fit into 7MB but can’t fit into 4MB.
I suspect the better latency at the high end is because you no longer have to work through the caching logic (which does little in such large transfers anyway). And I guess their bet is that you're not going to make many transfers that happen to fall into that window. That seems like a good bet, my intuition is that there's a "bathtub" style distribution here where accesses are either reasonably random (and thus don't involve transferring more than ~3 MiB at a time) or they're straight-through and involve very large sequential transfers (greater than 7 MiB).
Like the article notes, it seems to be an interesting decision. They speculate that the L3 cache on the A9 was actually a decision made for reasons of power efficiency (it takes less power to hit cache than to hit memory) and that wasn't a concern here, whereas they would have needed even more cache for the extra GPU cores, and die size was getting to be a concern. But they also note that the L3 cache didn't consume much space anyway, which is the weird thing to me.
So higher bandwidth and many uses cases that involve streaming (like say 4k video or writing triangles from the GPU to the frame buffer) it makes sense to get rid of the L3 and upgrade the bandwidth to 50GB/sec.
Hopefully Intel will match the bandwidth with their next generation. So far Intel's taken the easy route and added an off chip, but inside package cache for their highest end GPUs (Iris Pro) but left the memory bandwidth the same (128 bit @ DDR4 speeds around 25GB/sec.)
(I speak as one who loved sim_g4 as a modeling tool for verifying compiler efficiency, or rather, occasional lack thereof. So easy to see when a cache miss would occur, or just execution unit contention)