Curious what will be the answer from Nvidia.
Notice how Google did not submit benchmark results for several tests.
[0] https://blogs.nvidia.com/blog/2018/12/12/record-breaking-mlp...
So a Cloud TPU chip is slower than a V100 chip? Seems like that's the answer.
I’m also curious what the is the performance of the network interconnects in the TPU pod. I couldn’t find it documented, though I didn’t look super hard.
[1] https://github.com/uber/horovod/blob/master/docs/benchmarks....