Are the cards you mention from 2015? Are they for gaming or servers? Do they use ECC? If not, that rules them out right away.
You can find the paper with the methodology, theoretical peaks and latency management at https://arxiv.org/abs/1704.04760
The first one, for the memory comparison, is indeed from 2015.
The second one isn't -- 2015 desktop/server GPUs didn't have good half-float operations yet, as that hadn't really been a market. However, the first-mentioned GPU from 2015 has 8.6 single-precision TFlop/s, which is also more than 2x higher than their comparison baseline for GPUs.
The gaming/server and especially the ECC thing is pretty moot. First, while I'm not sure what kind of server SKUs were available at the time, it hardly affects architectural results. Second, even market availability shouldn't matter much. They're Google. If they had wanted different SKUs in volume, they almost certainly could have gotten them.
I mean, it's clear that a special-purpose chip is going to beat anything else at a task like this. It's just odd that they apparently felt the need to make themselves look better than they really are when the result is impressive enough with a real comparison.