http://hothardware.com/ContentImages/NewsItem/37034/content/...
(That's a P100 server, obviously, but it's the same kind of design. The P100 module looks a lot like the V100 module.)
http://hothardware.com/ContentImages/NewsItem/37034/content/...
(That's a P100 server, obviously, but it's the same kind of design. The P100 module looks a lot like the V100 module.)
The point of my previous comment is that it doesn't make sense to compare an entire TPU module (containing 4x TPUs) to a V100 board.
The closest comparison to a TPU module (with 4x TPUs) would be a dgx-1 board, which contains the nvlink bus that you mention, but also contains 8x V100 boards, hence why in my previous post I said you should compare the compute performance of a Volta DGX-1 (8x V100s) to 2 TPU modules (8x TPUs).
At the end of the day, it is simple, in a given area of space, you can get more compute performance from provisioning that area with V100s (in the form of using dgx-1s) versus provisioning it with TPUs (in the form of using TPU modules).