Intel could have had an interesting play there since they have GPUs and they have the tech for a product between flash and DRAM performance.
Intel could have had an interesting play there since they have GPUs and they have the tech for a product between flash and DRAM performance.
Honestly, it seems like a perfectly plausible solution: With 20us latency, you'd be able to load 50k layers per second. You can get as much bandwidth as you need by putting more dies in parallel.
Qwen3.8-flash-next has 48x sparse-MoE layers, and you don't know which experts need to be loaded for each layer until the prior layer has finished. So that requires 48x 20us delays per forward pass. That admits more than 1000 forward passes per second.
With typical MTP acceptance rates, you'd be looking at maybe over 2k tok/sec as the theoretical upper bound. A big flash memory on the GPU would be great for n-gram tables as well.