No one is running cpu-only inference anymore. Deepseek flash / Qwen flash next et al only need ~12-24 gb of VRAM for the active experts, so anyone with a decent gpu + lots of RAM + server cpu can run them at okay speeds.
Or one can also run Strix Halo / DGX spark / Apple. They handle MoEs well and use normal (not HBM) ram.
some benchmarks from locallama:
Deepseek V4: "~25t/s decode at full quant, no speculative decoding, 2x3090 (only one being used for this model) + 9684x w/ 12 channel ddr5 4800, latest llama.cpp"
"Getting about 300 tk/sec pp and 23 tk/sec decode on Mac M3 Ultra 256 GB. Using original full precision model weights."
Qwen-Flash-Next: https://www.reddit.com/r/LocalLLaMA/comments/1wom3fe/qwen_38...
https://www.reddit.com/r/LocalLLaMA/comments/1vcaztx/what_sp...