Hacker News Reader

Comment by aequitas | Hacker News Reader

> The simplest strategy is to just never invalidate. Some data doesn't go bad. The contents of CSV upload #42345 won't change. Neither will the HTML conversion of some markdown-formatted product data.

Thats a dangerous assumption to make and while simple at the cache's end might need complicated logic at the frontend. There will always be that use case of a customer that needs the file changed or removed. So instead of invalidating you now need a system to manage references to versioned cached objects.

View on HN

Yes. One danger of infrequently-invalidated or never-invalidated data is that it can turn into a ticking time bomb.

For example you may introduce a bug in the code path for calculating the fresh value that causes an error.

That code may not execute for months and months since the cache is warm and up and running. Until one day it is restarted and you find out the problems it was masking.

Pro tip: don't get greedy with cache TTL, define and enforce a low value that you can tolerate. In many use cases you can easily afford to re-compute once per 5 or 10 or 30 minutes.

We've given up on this whole idea and basically gone for extremely long TTLs (30 days) on our caching layer. We then built the tooling to dump caches after deploys and manually on request. You can dump caches at the application layer, or even down to the specific request param combination layer. It's worth investing a bit into systems.

I have seen systems that use caching so heavily that when the cache is invalidated en masse, they need to bring up extra compute resources to fill it again, because the system under normal load with an empty cache needs so many more resources. Basically the system can no longer function adequately with a cold cache. This is of course a pain to orchestrate.

I suppose regularly clearing out the cache is a way to be sure you aren't in that situation, or at least know it when you become so.

My experience with those systems is one reason that I'm reluctant to take the position of resorting to caching before trying to optimize the underlying code. It makes me want to treat caching as a last resort when I can't feasibly (whether due to time or skill or actual external limits) optimize any further, which is I think a bit different than the attitude OP is suggesting.

The maintenance "cost" of adding a cache layer is in my experience higher than the OP suggests. Failure-to-invalidate/stale cache bugs can be very easy to make and difficult to debug and solve when using a strategy that depends upon invalidation, like 2 and 3 in the OP.

If every file upload creates a new upload number (which is a common design) and the upload numbers are not exposed to the client then changes and deletions work without any special effort. I guess this is the case the author is talking about.

In many cases, such as the CSV one, I think the answer is converting the article's Strategy 1 to the article's Strategy 4 -- cache under a key that has a digest hash of the content in it, so when the content changes, the cache key changes.

If the requestor only knows the filename, now you have to cache the filename to hash lookup.

Then you've just moved the invalidation problem around.

This is a great strategy if the object is controlled by a build process, so your client code already knows what the hash is.

> Now you have to cache the filename to hash lookup.

The OP's "Strategy 4" is basically exactly that, although they describe it a bit confusingly as "namespacing" assuming a sort of hieararchy that we don't have here, without realizing it's more general purpose.

The OP acknowledges that in Strategy 4 "Each logical cache lookup now requires two real lookups - first for the namespace key [in our example lookup the filename to get the digest] and then for the actual key [use the digest to look up the actual content]"

I have more commonly used it when you can have the digest at the ready though, when you can make the requester aware of the digest. The "build time" doesn't necessarily have to be the software build time; I have sometimes put the digest in the database (which the requester has access to, and where it's cheaper to access then the entire content would be), which can then be used to fetch the content from the "cache". I guess this really is two cache lookups across two cache products, considering the database as a cache in that case, but sometimes the database fetch is going to happen naturally anyway.

You can just overwrite the cache with new information when the user replaces a file.

That's update on write, not never invalidate.

You can't, because you design to never invalidate, there is no way to ensure if and when this change is propagated through your entire cache layer (and local caches like browsers). And thus guarantee to the client the change has been made.

I'm not saying it's a bad idea. But often a seemingly simple solution to a complex problem doesn't take the entire complexity of the problem into account. Which is fine if you can consciously make that tradeoff but more often than not will turn into a footgun eventually.

Reply on news.ycombinator.com