This is calculation is pretty pointless and the title is flat out wrong. It also gets lost in finer details while totally missing the bigger picture. After all, the original paper written by people either working for Google or at Google. So you can safely assume they used Google resources. That means they wouldn't have used H100s, but Google TPUs. Since they design and own these TPUs, you can also safely assume that they don't pay whatever they charge end users for them. At the scale of Google, this basically amounts to the cost of houseing/electricity, and even that could be a tax write-off. You also can't directly assume that the on paper performance of something like an H100 will be the actual utilization you can achieve, so basing any estimate in terms of $/GPU-hour will be off by default.

That means Google payed way less than this amount and if you wanted to reproduce the paper yourself, you would potentially pay a lot more, depending on how many engineers you have in your team to squeeze every bit of performance per hour out of your cluster.