https://www.nvidia.com/en-us/data-center/a100/ is the most official reference. If you scroll to the bottom, you’ll see that an A100 part can do ~20 Teraflops (either FP32 or FP64 in little-matrix aka tensor mode). When they say “each A100 can do 5 petaflops”, they mean each DGX which has 8 such cards and thus they mean 600-ish something-ops per card. The generous assumption is that’s FP16 or bfloat16 for sparse ops, and therefore they are “flops”.
The reality is that if someone says “supercomputer” in the general sense, they mean scientific computing and so mean a double-precision LINPACK benchmark. The 1120 A100 parts (8x140) doing 20 “real” teraflops each has an absolute peak of about 22 Petaflops (and older code without tensor mode would be half that on FP64).
tl;dr: ML isn’t scientific computing, and those are different flops, but “10 petaflops” just doesn’t sound as impressive.