Hacker News Reader

Comment by brightball | Hacker News Reader

Depends on how large the cache is and how much users expect instant updates to their data. If real-time or near real-time is the expectation, then avoiding the cache usage unless necessary preserves their experience without incurring the hurdles of cache invalidation that grow more complex the more you cache.

EDIT:

Another area where this approach is beneficial is with a multi-tenant system where larger tenants querying more data could have a much more significant performance impact than other tenants on the system.

Time of day/week/month usage spikes can also cause a performance degrade for everyone even if it's not typically stressing the system.

This approach is just a mitigation to prevent the system from getting overload during those times. It's not the right approach for every situation. Just a spin on the Circuit Breaker Pattern.

https://en.wikipedia.org/wiki/Circuit_breaker_design_pattern

View on HN

I've always been a bit of a fan of debounce for this but many of the obvious implementations are cache based which is essentially inviting people to do the thing I just said I hate.

The cost of counting upvotes or logged in users is not worth the incremental value it affords the user. These are examples of situations where 'eventual consistency' can be embarrassingly easy. If you split the source of truth (SoT) from the system of record (SoR), then the cost of updating the 'truth' can be amortized across the writes to the SoR. The reads do a straight lookup from the SoT, which was the truth at some point but is now the fiction everyone is agreeing to. The distinction is that everyone agrees to the same source of truth. Nobody ever goes asking the SoR for a second opinion, except the SoR, which updates the SoT when appropriate. The SoT is dumb, the clients are dumber, only the code that manages the SoR has any smarts to it.

Read traffic can escalate pretty high in such a system without much stress.

Exactly. Another situation is a multitenant system where some tenants are much larger than others, triggering larger queries which potentially have a larger performance impact.

I'm beginning to wonder whether my personal moniker of "signal propagation" is nominatively equivalent to what others call "cache invalidation".

I.e. you cannot by definition have a state change until the signal thereof has traversed a link from A to B. No signal, no change. If you have a cache for one system that delays writes to a backing data store referenced by other systems that are unaware of that cache, then you have necessarily a partition problem.

Every problem in a distributed computing context can generally be reduced to a signal propagation problem.

It's only a problem if you're breaking deadlines. Like having someone's status in communication app update with 10s delay is probably not a a problem, so as long as you don't ever cache it for longer than that, it is entirely fine.

> If you have a cache for one system that delays writes to a backing data store referenced by other systems that are unaware of that cache, then you have necessarily a partition problem.

In most cases people mean caching reads so the problem here is stale reads, not delayed writes. It's not necessarily the same, some component caching (or essentially, queuing) write means that once the write is done, all readers (assuming they don't have any extra cache) see it at once, while with caching reads (and no invalidation), multiple readers can have different view

The one that really jams me (and others) up is that signal propagation and transaction lifetimes line up badly. Caches, being cheap imitations of ACID 'compliant' databases, which themselves cheat all day every day to make pretty numbers, almost never have read repeatable semantics, so if your code splits responsibility for fetching a piece of data you have one of the problems I complained about at/near the top level: Half of a decision can be made with one version of a piece of data and the other half with another, resulting in undefined behavior. In a system that asserts that A = !B you can have an interaction that did some of the activities for A and some of the ones for B (because we read A and A' and assumed they were identical).

You're making a lot of assertions without any backup, and they don't make much sense.

> Caches, being cheap imitations of ACID 'compliant' databases

No they're not, not remotely.

> which themselves cheat all day every day

I don't believe a word of this, have you any actual evidence (other than I believe Oracle's serialisable is actually snapshot isolation IIRC)

> Half of a decision can be made with one version of a piece of data and the other half with another, resulting in undefined behavior. In a system that asserts that A = !B you can have an interaction that did some of the activities for A and some of the ones for B

And this just makes no sense, you're claiming that databases are not producing the results they should, if that's the case then I don't believe you understand transaction isolation levels or indeed how database queries work, in which case you have problems other than and larger than caching.

This whole thread s just getting weirder.

As someone said in another subthread:

> That's with everything that have any level of complexity, just see how many people got transaction isolation wrong just because they had some simplistic view on how it works in database they use.

Any database without MVCC works quite a bit differently than people think it does. And ones with MVCC run into the ceiling if you assume they work like databases that aren't based on MVCC. Like leaving long transactions running in the background.

The problem is, like caching, difficult to catch in a petri dish.

> This whole thread s just getting weirder.

Maybe you should take a break.

If you assume your database isn't doing much for you, you

WTF?

> If real-time or near real-time is the expectation, then avoiding the cache usage

Because looking up a precalculated result in a hash table is so much slower than not looking it up? And what, recalculating it instead is faster than an O(1) hash table lookup? If so,why are you caching?

I am just getting more confused by what people are saying here.

I do think you've hit on a point of confusion in the parent, but...

In a (hard) real-time system the only thing that matters is the worst-case execution time -- if your worst-case time fits in the allowance you win, if it doesn't you lose.

Adding a cache improves the average case (typically), but hurts the worst case (always) -- your worst case goes from "do expensive operation" to "do lookup in cache, fail, do expensive operation." Even if you parallelize misses ("do lookup in cache | do expensive operation, on cache success cancel expensive operation, on cache miss wait for expensive operation" you're generally adding at least a few operations in the critical path.

You raise a good point but I would never have thought caching appropriate to an RTS, not that I have any experience with those. But yes.

> I am just getting more confused by what people are saying here.

Maybe the takeaway is that this shit is hard and everyone is a little bit confused in their own way.

I'm not saying the only way to win is not to play, but I can neither confirm nor deny rumors that I may have at some point muttered this under my breath.

You can queue cache refresh in the background if the object in cache is close to the "limit" of how fresh you want it to be. Need a bit of cleverness to avoid thundering herd problem but otherwise works pretty well.

Also if you really want near-real time you might want to actually push the updates directly to cache instead of relying client to trigger cache filling. cases where you need a ton of data and it served real-time and it's same object that constantly changes (and not say pointer to a position of video stream) are pretty rare

My problem with the term 'cache' here is that people have expectations about what cache is and when you violate them you tend to discover that they've returned the favor.

> push the updates directly to cache instead of relying client to trigger cache filling

You can use cache software to fulfill such a role but it's not exactly a cache anymore, and keeping people from treating it like one is some kind of cat herding.

You're using it as a data store at this point, which for one has very different eviction semantics - exactly 1 copy of each concept, no more, no less. You can store such things in Consul, for instance, and get very low stale read latency. The cache implementation scales higher but breaks sooner. Pick your poison.

Well, it depends on your data model. Like, if you write only a little (compared to reads), write-thru model of cache (write to cache as well as to backing store) will lose you tiny % of hit ratio at profit of never having to worry about stale data.

> My problem with the term 'cache' here is that people have expectations about what cache is and when you violate them you tend to discover that they've returned the favor.

That's with everything that have any level of complexity, just see how many people got transaction isolation wrong just because they had some simplistic view on how it works in database they use.

Reply on news.ycombinator.com