Why so little L3 cache on the competition?
As such, AMD can make absolutely huge amounts of L3 cache (well, many parallel L3 clusters), while other CPU designers need to figure out how to combine the L3 so that a single core can benefit it from it all.
The first generation of Epyc had a complicated hierarchy that made latency quite hard to predict, but the new architecture is simpler. A CPU can talk to a cache in the same package but on a different die with reasonably low latency.
(I don't have numbers. Still reading.)
Think of the MESI messages that must happen before you can talk to a remote L3 cache:
1. Core#0 tries to talk to L3 cache associated with Core#17.
2. Core#17 has to evict data from L1 and L2, ensuring that its L3 cache is in fact up to date. During this time, Core#0 is stalled (or working on its hyperthread instead).
3. Once done, then Core#17's L3 cache can send the data to Core#0's L3 cache.
----------
In contrast, step#2 doesn't happen with raw DDR4 (no core owns the data).
This fact doesn't change with the new "star" architecture of Zen2 or Zen3. The I/O die just makes it a bit more efficient. I'd still expect remote L3 communications to be as slow, or slower, than DDR4.
As a comparison, an IBM z15 mainframe CPU has 10 cores and 256MB per socket.
Well, that's eDRAM magic, isn't it? Most manufacturers are unable to make eDRAM on a CPU.
> My question is why they chose to have just 32MB for up to 80 cores when AMD can choose to have 32MB per 8-core chiplet.
From my understanding, those ARM chips are largely I/O devices: read from disk -> output to Ethernet.
In contrast, IBM's are known for database backends, which likely benefits from gross amounts of L3 cache. EPYC is general purpose: you might run a database on it, you might run I/O constrained apps on it. So kind of a middle ground.
> those ARM chips are largely I/O devices
Anything Neoverse-N1 is pretty good at general compute and databases. There's definitely a lot of Postgres running on AWS Graviton2 instances already :)
Also since Xeon dies are monolithic, unlike AMDs chiplet design, means that increasing the size of certain components on the die, like cache for example, increases the risk of defects wich reduces the yields, making them unprofitable.
I wonder if we could get numbers for L3 misses and cycles spent waiting for main memory under realistic workloads.
I suggest learning to read performance counters, so that you can get information like this yourself! L3 cache is a bit difficult for AMD processors (because many cores share the L3 cache), but L2 cache is pretty easy to work with and profile.
General memory-reads / memory latency is pretty easy to read with various performance counters. Given the amount of latency, you can sorta guess if its in L3 or in DDR4.
Thus, 'ye olde' would be pronounced 'the old'. 'Ye' is just a different form of 'the' from a time when 'y' represented 'th'.
So, 'the ye olde' would be read as 'the the olde'.
That said, there are two major reasons:
1. Epyc is on a chiplet architecture. Large chips are harder to make than small ones. Building two 200mm^2 chips is cheaper than building one 400mm^2 chip. Since Epyc has a chiplet architecture, this means they can put more silicon into a chip for the same price. This means that Epyc can be just plain bigger than the competition. This comes with some complexity and inefficiency but has, so far, paid off in spades for them.
2. Epyc is on a newer process. This means AMD can fit more transistors even with the same area. Intel has had serious problems with their newer processes, so this is not an advantage AMD expected to have when designing the part. The use of a cutting-edge process was, in part, enabled by the chiplet architecture. It is possible to fabricate several small chips on a 7nm process even though one large chip would be prohibitively expensive, and AMD has been able to use a 14nm process in parts of the CPU that wouldn't benefit from a 7nm process to cut costs.
The first point is serious cleverness on the part of AMD. The second point is mostly that Intel dropped the ball.
[1] https://www.anandtech.com/show/16021/intel-moving-to-chiplet...
[2] https://www.anandtech.com/show/16051/3dfabric-the-home-for-t...
The mobile market cares more about efficiency than easily scaling up to much bigger chips, so the M1 and other ARM chips are probably going to ignore this without much consequence for smaller chips.
Intel still tops sales because of non-perf related reasons like refresh cycles, distrust of AMD from last time they fell apart in the server space, producing chips in sufficient quantities unlike the entire rest of the industry fighting over TSMC's capacity, etc.