TPUs are slow but efficient and AMD has bugs but for some things works quite well. Nvidia is obviously the gold standard.
There are good 1p [a] and 3p [b] benchmarks comparing TPUs vs NVIDIA GPUs.
[a] - https://github.com/GoogleCloudPlatform/vertex-ai-samples/blo...