Hacker News Reader

Hi,

If you ever get to writing a blog post about kv cache quantisation, i'm interested in quantising K differently than V

If you like, you can run these tests yourself as well. It is ~$500 per a single combination, and assuming no failed runs.

I don't have anywhere near that kind of money to spend. Sorry.