Amazing article. I was under the misapprehension that temp and other output parameters actually do affect caching. Turns out I was wrong and this explains why beautifully.
Great work. Learned a lot!
Great work. Learned a lot!
Where do people get the idea from that temperature affects caching in any way? Temperature is about next token prediction / output, not input.
It´s a semantics issue where the word caching is overloaded depending on context. For people that are not familiar with the inner workings of llm models, this can cause understandable confusion.