A "single" EPYC is eight 4-core processors so there are even more NUMA effects there.
Or perhaps it has absolutely nothing to do with memory speeds, but I can't think of what else would have such a dramatic effect. I'd love to see the same test with a single Xeon.
In NUMA-speak, each group is (probably) a node with neighbouring nodes having a lower distance.