This comparison seems backwards? Also I have no idea what it means - it seems like really strange phrasing and I can’t seem to make sense of it. Does anyone understand what was trying to be conveyed here?
2,693 karma · joined February 8, 2017
This comparison seems backwards? Also I have no idea what it means - it seems like really strange phrasing and I can’t seem to make sense of it. Does anyone understand what was trying to be conveyed here?
Buried at the end there is the details I was most interested in - a possible competitor for DeepSeek V4 Flash? Excitedly awaiting the release of the weights for this one.
A few years ago I actually traveled all around the US and worked remotely with a 4g unlimited data sim card and was pretty amazed at the coverage, only very few places was I left without coverage. I'm guessing it's even better now?
This made me wonder if Starlink is a much more niche product than I thought - I actually thought it was significantly faster than cell data - but it seems it's just for folks in super remote areas at this point?
The 2 bit quants are really good. I have a lot of memory so I can squeeze it all in at ~80gb.
I do wonder if you are planning to eventually delegate some of the merging responsibilities? I see some other projects follow a similar pattern when they grow to a certain size.
When the DSpark paper came out[1] the next day we had folks attempting to implement, working together, validating their failures. Eventually their work being synthesized into a PR[1] that admits performance is not ideal. Something antirez alluded to in one of his videos (speculative decoding is a great boon, but mostly for large labs hosting and serving many requests at once, and maybe not so effective for local inference).
There's recent work into "directional steering"[3] that has made it's way into per-session directional steering overrides thanks to audreyt[4].
There's support for the new Hy3[5] model also thanks to audreyt[6].
There's Pre-M5 optimizations[7] in the queue thanks to ivanfioravanti who also helped with some of the initial M5 optimizations.
I haven't watched a repo like this since llama.cpp and whisper.cpp in the early days (though llama.cpp is pretty exciting right now with the SYCL improvements that are flowing in for the new Intel GPUs).
The DS4 repo is a really interesting place to watch folks who heavily code with agents collaborate together in a way that seems pretty effective. I've been really enjoying it.
[1]: https://arxiv.org/abs/2607.05147
[2]: https://github.com/antirez/ds4/pull/502
[3]: https://arxiv.org/html/2406.00045v2
[4]: https://github.com/antirez/ds4/pull/148
[5]: https://hy.tencent.com/research/hy3
Has anyone used the new Minimax M3 model? I’m curious how it compares with Deepseek V4 and GLM 5.2 and other larger open weights models.
It heavily utilizes ssd streaming from my understanding and I think he mentioned getting some semi usable speeds on a 128gb m5 mbp.
https://github.com/antflydb/antfly
They’ve put a lot of effort into optimizing the local llm pipelines and I have a lot of faith in the devs working on it.
What I think is really cool is that the search happens using http range queries across statically hosted parquet files.
I think things like this could bloom into a relatively open and distributed search ecosystem that isn’t controlled by major corporations.
It has the same 256 GB/s memory bandwidth limit as every board previously, not sure why this is even being released right now as if it's some new fangled thing - you can go get a Framework Desktop for roughly the same price or a GMKtec EVO-X2 for a bit cheaper.