Comparing SMs to CPU cores 1:1 also makes no sense. They don’t do the same things.
Comparing SMs to CPU cores 1:1 also makes no sense. They don’t do the same things.
For FP64 computations, the reverse happens, a 9950X buries a 4090, despite the latter having a 3-times higher price and a 2.5-times higher power consumption.
For FP64 operations, 4090 and 9950X are able to do a similar number of operations per clock cycle (288 vs. 256), but 9950X can do them at a double clock frequency and it is easier to reach a high fraction of the maximum theoretical throughput on a 9950X than on a 4090.
FP8 is a more recent quantization format and AFAIK no CPU implements it.
I do not know which is the throughput of these instructions for Zen 5. It must be higher than for older CPUs, but it must be slower than for the Intel Xeon models that support AMX (which are much more expensive, so despite having a higher absolute performance for inference, they might have lower performance per dollar) and obviously it must be slower than for the tensor cores of a big NVIDIA GPU.
Nevertheless, for models that do not fit inside the memory of a GPU, inference on a Zen 5 CPU may become competitive.