Hacker News Reader

I'm not (currently) objecting to the actual MLPerf results.

I'm objecting to the fact that your marketing team (at least, I hope it was your marketing team) chose to represent 5 different GPU systems as the same color and 3 different TPU systems as another color in one bar chart.

There's a book with a chapter or two on how to reasonably compare the performance of computer systems. Given that the author (Patterson) works on TPUs, I would expect better.

View on HN

Thanks for the feedback. As more and more new ML system architectures emerge over the next few years, these performance comparisons will get even more complicated. Chip-to-chip comparisons and server-to-server comparisons are arbitrary, and it will sometimes be difficult to define these boundaries at all.

Our impression is that top-line performance comparisons independent of system size (like the comparison in the blog post) and performance-per-dollar comparisons are the easiest to understand, but we're certainly open to other ideas.

> top-line performance comparisons independent of system size

Then do that, just don't do it all in one chart with the systems only specified in the fine print.

Reply on news.ycombinator.com