Cache Organization in Intel CPUs (2009)
duartes.org
duartes.org
Because a 4 KB page has 64 cache lines, then you can have at most 64 cache sets. With an 8 way associative cache this works out to 32 KB. Using 128 sets would cause aliasing, but with 64 sets the cache index is built from the LSBs that just index into the page (i.e. not used in the TLB lookup). Thus, the only way to grow increase L1 capacity is to: - totally abandon 4KB pages in favor of (e.g.) 2MB pages (not likely) - increase cache associativity (likely imo) - stop using virtual index+physical tag (not likely imo)
On an unrelated subject, do you know if the next desktop generation (coffeelake?) will support AVX512 or should I just buy a skylake-x?
https://www.kitguru.net/components/cpu/anton-shilov/intel-sk...
I doubt we'll see cannon lake this year, since they released kaby lake early 2017...
https://en.wikipedia.org/wiki/Coffee_Lake https://en.wikipedia.org/wiki/Cannonlake
Increasing cache size grows latency in two ways: every doubling of the cache size adds one mux to the select path, and every doubling increases the wire delay from the most distant element by ~sqrt(2). Both of these are additively on the critical path, and spending more time on them would require increasing the cache latency.
The size of a cache is always a tradeoff against the latency of the cache. If this was not true, there would only be a single cache level that was both large and fast. However, making something both large and fast is impossible, so instead we have stacked cache levels starting with a very fast but small cache followed by increasingly slower and larger ones.
By your reasoning, no cache should be able to grow, because then their latency would increase too much. But instead, all other CPU caches are growing basically with iso-latency. The reason this is possible is technology scaling. Anyway...
But yes, the L1 does have to be small and fast, but it doesn't have to be that small to be that fast. It has to be that small because of virtual indexing combined with the cost of adding ways breaking other design constraints (possibly a latency constraint, fine). But you could grow the L1 by adding sets and get your required latency.
The problem is that technology scaling only gives you roughly enough to keep up with the speed of the CPU. Cache latencies measured as nanoseconds keep going down, but cache latencies measured in clock cycles are pretty stagnant at the same sizes. And when Intel added some more L3, they also relaxed latency to it, and when they recently cut the latency to it a little, they did so by cutting the amount.
> But yes, the L1 does have to be small and fast, but it doesn't have to be that small to be that fast. It has to be that small because of virtual indexing combined with the cost of adding ways breaking other design constraints (possibly a latency constraint, fine). But you could grow the L1 by adding sets and get your required latency.
No, you couldn't. The added latency of increasing the size would be simply too much. I know for a fact that the L1 load latency is currently one of the most important critical paths in Intel CPUs -- any increase in L1 size would mean that you have to reduce clock speeds.
Design Principle 2: Smaller is faster. [1]
BTW, if you look at Agner Fog's latency tables [2], mov mem,r (load) went from 3 cycles in Haswell to 2 cycles in Skylake. So Intel has been concentrating on faster which is nice.
And by way of comparison, AMD increased their μop cache size in Ryzen but then only slightly. Way size went from 6 μops to 8. This matches their increase in EUs.
[1] Patterson and Hennessy. Computer Organization and Design, 5th edition, p. 67.
Always impressed that Agner Fog takes the time to publish his results. Pretty amazing. But I think focusing your thinking on the register count in MIPs or the the uarch for some random opcode does not get into the real constraints on L1 cache design at all. One could say that x86 should be even faster, because hey, far less than 32 registers (or historically at least).
My response is like this: yes, the L1 has to be small to be fast, but it has been stuck at 32KB forever now. It could have grown! So it's not as simple as small is fast.
I also vaguely remember the Mill cpu guy talking about cache size constraints just because of the speed of light, but given node size has continued to decrease during the last decade while frequency has nearly stopped to increase, this might be less an issue than basic area optimizations. Or this might be an interesting consideration on Mill only because it is a radically different architecture, and needs different area ratios.
Only wild guesses though, I don't even have tried to confirm any of that with any kind of research or back of the envelop calculations.
I think at 6 transistors per bit we really aren't talking about a lot of die area. Still I'm stone cold certain the Intel architects would increase L1 cache size if that was beneficial, if it modeled out. (However they may want to keep performance similar+predictable unless there's a solid win.)
Agner is showing they've reduced L1 latency. So this smaller is faster seems to have gotten them something.
So you really have to work backwards and ask why they didn't/don't. There may more than one reason; but they don't and haven't in quite some time.
I'm old school assembly/compiler hack. I read Agner and the Intel Optimization Manual a lot. VTune, IACA and the PMCs. Someone has to do it.
It's far easier to add capacity by adding sets, as opposed to ways. But they can't add sets in the L1 because of the aliasing problem. When they do increase L1 capacity, if nothing else has changed, then it will be by adding ways.
BTW, renamed != ROB. I got that wrong above.