AMD has a Radeon Pro SSG card with 16GB + 2TB on board memory.

You address the 2TB NVMe SSG memory as a filesystem directly from the card itself - no PCIe system transfer involved. 24.6TFLOPS half-precision.

You can program using OpenCL or HIP (opensource cuda replacement )

https://pro.radeon.com/en/product/pro-series/radeon-pro-ssg/.

https://github.com/ROCm-Developer-Tools/HIP