Hacker News Reader

Is there some cost rule of thumb to compare Nvidia, AMD, and Google TPU?

Good question. No good metric give performance depends on software stack (JAX vs PyTorch) + optimizations.

But my take performance per dollar of TPU > AMD > NVIDIA.

TPUs are slow but efficient and AMD has bugs but for some things works quite well. Nvidia is obviously the gold standard.

Haha, TPUs are not slow :) All of Google's training (including Gemini models) is done on TPUs.

There are good 1p [a] and 3p [b] benchmarks comparing TPUs vs NVIDIA GPUs.