A 64 bit float operation is >4X as expensive as a 16 bit float operation.
Most parallel research, especially at this scale is more about different balance of operations to memory bandwidth, and much more worried about interconnect latency.
I wouldn't assume that just because various corporations have large training clusters that they could dominate HPC if they wanted to. Hyperscalers have dominated throughput for many years now, but HPC is a different beast.