Comment by pshc | Hacker News Reader

Top New Best Ask Show Jobs

Comment by pshc | Hacker News Reader

Parent Full thread

Only 30 tok/s on Qwen 3.8 Flash Next? These days thanks to those specialized engines out there I can get ~2x that speed on a 24 GB video card with RAM offload. Thermal limits, I assume...

I wonder how its performance is on dense models.

memeory bandwidth is typically the limiting factor, not compute. Your card quite likely has 2x or more the bandwidth.

Yeah, its pretty low. I get about 60tps with a specialized engine on Strix Halo that runs UD-IQ4_XS with MTP

Which engine?

Halogen

Which video card? Which specialized engine?

Reply on news.ycombinator.com