Hacker News Reader

With that amount of memory can you run 4-bit DeepSeek 4 Flash? It is way more efficient in the KV cache department so may be worth a try

I haven't looked into DS4 yet but based on antirez's results on 128 GB Macbooks, it shouldn't be a problem to run it on a pair of RTX6000 Pros.