DRAM speeds is one thing, but you should also account for the data rate of the PCIe bus (and/or VRAM speed). But yes, holding it "lukewarm" in DRAM rather than on NVMe storage is obviously faster.
In general systems usually have PCIE version with bandwidth better than RAM of that system.
For example a system with DDR4 (27Gbs) usually has at least PCIE4 (32Gbs at 16x).
But you can bottleneck that by building a DDR5 (40Gbs) system with PCIE4 card.