KV cache compression, so how much memory the model needs to use for extending its context. Does not affect the weight size.

I wrote this more intuitive explanation. I think you might find it helpful!

https://prabal.ca/posts/google-long-context-cheaper/