Hacker News Reader

Which quant of QFN are you running on your Strix Halo?

HF is telling me that IQ4-XS is the largest one likely to fit without much contortion.

That's the one I'm running using a fork of the gufo inference engine as gufo upstream doesn't support that quant.

I'm running a regular Q4, what about that quant format justifies the fork?