Has it been tried on 128GB M4 MacBook Pro? I'm gonna try it, but I guess it will be too slow to be usable.
I love the original DeepSeek model, but the distilled versions are too dumb usually. I'm excited to try my own queries on it.
I love the original DeepSeek model, but the distilled versions are too dumb usually. I'm excited to try my own queries on it.
I love the original DeepSeek model, but the distilled versions are too dumb usually.
(I've been using the 32B and while it could always be better, I'm not unhappy with it)
Using LM Studio, trying to load the model throws an error of "insufficient system resources."
I disabled this error, set the context length to 1024 and was able to get 0.24 tokens per second. Comparatively, the 32B distill model gets about 20 tokens per second.
And it became incredibly flaky, using up all available ram, and crashing the whole system a few times.
While the M4 Max 128GB handles the 32B well, it seems to choke on this. Here's to hoping someone works on something in-between (or works out what the ideal settings are because nothing I fiddled with helped much).
In theory half of the model fits to RAM, so it should be GPU limited if memory management is smart.