Tearing Apart Google’s TPU 3.0 AI Coprocessor

The TPU 2 was about half the cost of using Nvidia for same work.

Be interesting how much further ahead Google is now with the 3.

But most impressive is able to offer Wavenet at a competitive cost to the old technique for TTS used by everyone else.

16k through a NN in real time is just hard to believe possible. Nvidia has their work cut out for them.

Their version of fp16 (bfloat as they call it) is very interesting. 7 bits of mantissa is only about 2 decimals of precision!

This is a very interesting read. I wonder if Google has an in-house fabrication facility for all of its TPU's.

They don't; they probably use TSMC.