Hacker News Reader

Thanks for detailed analysis, would be curios to see FP8 comparison too, given vllm has some custom kernels

Unfortunately, neither VLLM nor TGI support FP8 on AMD yet. But once they do, we will look into it.