DS4's architecture was redesigned for cache efficiency. It's the model itself that is just way more cache efficient, in addition to a distributed KV cache (which they open sourced last year). You could run your own LLM provider just for DS4 models, recreate their setup, and get the same cache efficiency, but it would only work for that one model family.

Other models would also need to be rearchitected to get the same benefit, and I suspect they don't want to, because it seems to create both larger models and slightly less intelligence (DS4* has smaller active experts to get faster results).