Hacker News Reader

Comment by porphyra | Hacker News Reader

I feel like a lot of the big GPU clusters that companies like Meta are using would score highly on the benchmark, even though Linpack seems to be based on fp64 which the H200 is kinda sucky at. However, there's only one GH200-based machine (Alps, with 2688 GH200 nodes) in the top 10 even though I'm fairly sure it's quite a bit smaller than whatever Meta is training Llama 3 on. Is there any reason why we don't see those show up in TOP500?

View on HN

The machines that the large ML companies are using are typically not going to run TOP500 competitively. They tend to skip the MPI stack, and weren't optimized for LINPACK performance. When I worked on TPUs there was no interest in attempting to benchmark them using supercomputer workloads (probably a good thing).

> I feel like a lot of the big GPU clusters that companies like Meta are using would score highly

Some might, but most of the work done on GPU compute was subsidized by DoE Exascale projects like Aurora, like my anecdote about the DGX product line above.

They are busy running training/inference on them and haven't benchmarked them?

So these benchmark lists like TOP500 are just fundamentally flawed because the serious players are too busy doing actual work than to participate?

I mean there is no rule if you buy a supercomputer that you have to benchmark it and submit the results. This said, in the days before AI the number and type of players that had this amount of compute were also commonly the types to submit their scores to said benchmark lists.

Most players that had this amount of compute also tend to pay less than the equivalent corporate wage, and so besides being a good way to stress-test your cluster (prior to general availability), it gives you the positive feeling of “I helped make this, I help run this.”

I work with a couple of the national labs and the talent there is generally incredible.

"I went to school for > 25 years so my life's work could be selling more ads" isn't a motivator for most of these people.

Thanks much for the info! Sentiments like that make me hopeful for humanity.

It is a good advertisement for interconnect vendors.

TOP500 and other similar HPC benchmark lists are only for FP64 computations.

While both NVIDIA and AMD design their top GPU model for both FP64 and AI/ML workloads, to save on the design cost, you can do AI training using GPUs that have only high AI performance (like RTX 4090 or its workstation counterpart, RTX 6000), without implementing FP64 operations at all (the FP64 performance of RTX 4090 is negligible, being worse than of any decent cheap CPU, it is provided only for compatibility in testing).

Or have them set up in a way that makes them hard to run full-system benchmarks on. I can think of a couple of financial firms that have clusters that would rate on the Top 10, but a) as you suspected, they’re too busy running the money printers to take them down for a few days to run benchmarks, and b) they’re split up into more manageable little clusters, so the high speed fabrics don’t connect and allow every node to talk to every other node, which you need for an HPL run.

Are you willing to dump a name or two of those finance firms?

A single 8xH100 node hits 15.6 fp16 petaFlops

Reply on news.ycombinator.com