It was trained on one NVIDIA GB200 NVL72. This is a tech demo!
You undersell. One of those is equivelent to like one hundred seventy H100s or four hundred+ RTX 5090s .. 13TB of GPU mem at 570TB per second bandwidth and 360 petaflops of FP8 tensor compute.