No, the AMD ones he’s talking about allow the GPU to address that memory as well: they’re “APU’s” that share system RAM as GPU memory
That said the compute power of the GB10 is awesome, my DGX Spark-like has been better than I expected.
That said the compute power of the GB10 is awesome, my DGX Spark-like has been better than I expected.
Not everything with AI is LLMs running on llama.cpp.
The massively lopsided compute makes speculative execution techniques (MTP/DFlash etc) way more effective, which makes modern MoE models quite usable
I’m running Qwen 3.8 Flash Next with Nota AI’s conservative NVFP4 quant at 30tk/s lowest decode, usually more around 45-55 for coding, and with prompt processing at a solid 2000tk/s
And that’s without very optimised kernels or anything, mostly standard vLLM