S, I'm running one of the unsloth quants. Q4 something K_XL I think?
I don't think it's all resident in memory on the 3090 but it runs fast enough for use, I'd have to check but it's like 30 t/s or so, fast enough that it doesn't bother me.
I don't think it's all resident in memory on the 3090 but it runs fast enough for use, I'd have to check but it's like 30 t/s or so, fast enough that it doesn't bother me.
No comments yet.