Fabric-wise, PCIe 4.0 offers 16 Gbit/s per lane, so a single PCIe x16 device would have 256 Gbit/s of bandwidth available (in each direction), and a single server could reach 1 Tbit/s with four NVMe devices on four PCIe slots. There is the overhead of 128b-132b encoding to be considered, but that's minimal (3%). There is also the overhead of PCIe packets to consider, but if the NVMe device supports large maximum payload sizes (1kB+) it should be minimal.
Are there PCIe 4.0 x16 NVMe devices commercially available?