> "The reason why we took [NVLink] out was because we needed the we needed the I/Os for something else, and so, so we use the I/O area to cram in as much as much AI processing as we could," Huang confirmed and explained of NVLink's absence.
> For gamers, creators, and workstation users who want a multi-GPU setup with the latest RTX 4000 series, Nvidia is switching from NVLink as a bridge to the PCIe Gen 5 standard.
PCIe 5.0 x32 supports 128GB/s.
"PCIe 7.0 Draft 0.5 Spec: 512 GB/s over PCIe x16 On Track For 2025" https://news.ycombinator.com/item?id=39933959
> The HBM3E [RAM] Wikipedia article says 1.2TB/s (2024) https://news.ycombinator.com/item?id=39938652
"Fiber-optic data transfer speeds hit a rapid 301 Tbps" (2024-03) https://news.ycombinator.com/item?id=39864107
HPE Cray Slingshot is a copper Ethernet HPC interconnect: https://www.nextplatform.com/2022/01/31/crays-slingshot-inte... ... "HPE Slingshot Makes The GPUs Do Control Plane Compute" https://www.nextplatform.com/2022/08/15/hpe-slingshot-makes-... :
> Here’s the gist. With normal GPU aware MPI software, inside the node the GPU vendors – the only two that matter today are Nvidia and AMD – have their own mechanisms for doing peer-to-peer communications between the GPUs – in this case NVLink and Infinity Fabric. For MPI data exchanges between GPUs located in different nodes, which is often required for running large simulations and models as well as for large AI training runs, MPI data movement is done through Remote Direct Memory Access methods, pioneered with InfiniBand adapters decades ago, which allow for the transfer of data between a GPU and a network interface card without the interaction of the host CPU’s network stack. [...]
> In a [recent paper], HPE showed off the new MPI offload method, making the distinction between the typical MPI communication that is GPU aware versus the stream triggered method that is GPU stream aware.
"Exploring GPU Stream-Aware Message Passing using Triggered Operations" (2022) https://arxiv.org/abs/2208.04817 https://scholar.google.com/scholar?cites=1354761056738606744...