How does one test the latency and bandwidth of CPU cache at each level?
Rdtsc is the read timestamp counter instruction. Every .25nanoseconds, that counter goes up by 1 (assuming 4GHz CPU)
Write 32kB of data and measure how many nanoseconds pass. Done for L1 cache.
Do the same for L2 cache size.
Then do the same at L3 cache size.
Finally do it for 4GB or something that only fits in main memory.
With retirement queue sizes ranging from 300 to 600 (depending on architecture), your singular CPU-core could very well be "executing" 600+ instructions simultaneously, including "rdtsc".
Under such conditions, the "definition" of "get the current time" is quite ambiguous.
-----------
This isn't a problem with bandwidth. But it makes measuring "latency" very difficult, especially because out-of-order execution means that "proper" latency is largely ignored.
-------
Because of the out-of-order execution models of modern processors, "reciprocal bandwidth", or "instructions-per-clock" tends to be more important than latency proper.