Hacker News Reader

Why is 4090 so slow in your tests? It regularly demolishes my 3090 in my own tests (2x speedup in most tasks).

LLM decoding is dominated by memory bandwidth, and 3090Ti and 4090 happen to have the identical theoretical memory bandwidth