Hacker News Reader

There's a glimmer of hope with ROCmFP4 which seems to double the current throughput: https://github.com/charlie12345/rocmfp4-llama

View on HN

I saw that but end of the day, the chips themselves don’t have hardware support for FP4. There’s smart ways around this limitation but it will never natively be close to true FP4 performance like MXFP4 and NVFP4 (happy to be proven wrong though).

Reply on news.ycombinator.com