time to first token != tokens per second
remember that EU -> US is ~150ms unavoidable latency, for example. then your comparison is local H100 vs Grok + 150ms latency to first token.
remember that EU -> US is ~150ms unavoidable latency, for example. then your comparison is local H100 vs Grok + 150ms latency to first token.
I said "and extremely low latency" because I know they are different. Groq's TTFT is still consistently competitive with any other provider, and lower than most of them. Here's some benchmarks: https://github.com/ray-project/llmperf-leaderboard#70b-model...