Cache is a huge component of "high priority" costs because it essentially forces the GPU to be idle (holding earlier prompt text in GPU memory) while waiting for newer input tokens. When an agentic session is active, this is of course much more preferable compared to loading in hundreds of thousands of input tokens again and again on different GPUs, but "walking away from the laptop" while waiting for the cache to expire is something that royally sucks today for inference providers. (Caveat: this is my layperson's understanding, I don't work in the space, I'm likely not up to date on best-practice caching practices)

"Low-priority" inference tasks like that, though, would never need to be cached. Again, who cares if it takes 12 hours? If you didn't initiate the prompt, then what do you care? You just randomly get notified when something is already ready for you to take a look at.