Hacker News Reader

25% MFU :( maybe because of the P2P nerf?

Maybe get a 7900 XTX. 122 TFLOPS of BF16/FP16 for less than $1k and I'm getting 55.4% MFU

These are not apples to apple comparison, as this is running across GPU and much bigger model

This much bigger model (500M), P2P is enabled via Mailbox. It is expected because of memory to compute ratio

can you elaborate?