The interesting part is this is done in Pallas!
Seems like the Pallas of old has completely been upgraded
Seems like the Pallas of old has completely been upgraded
What's interesting is that the MGPU team has achieved SOTA Blackwell GEMM performance before Triton (which IIUC is trying to bring up Gluon to reach the same level). All the big players are coming up with their own block-based low-level-ish DSLs for CUDA: OpenAI, NVIDIA, and now Google.