Hacker News Reader

Hm, that’s doubtful. I think even one instance of DS 4.1 Flash alone at ~500B parameters needs a full node (8 GPUs).

4xh200 seem to be enough if you cpu offload the engrams (which is pretty typical and even expected).[1]

And these are quite old cards by now. But yeah "all of OpenRouters traffic" is gonna take more than a rack