> If real-time or near real-time is the expectation, then avoiding the cache usage
Because looking up a precalculated result in a hash table is so much slower than not looking it up? And what, recalculating it instead is faster than an O(1) hash table lookup? If so,why are you caching?
I am just getting more confused by what people are saying here.
In a (hard) real-time system the only thing that matters is the worst-case execution time -- if your worst-case time fits in the allowance you win, if it doesn't you lose.
Adding a cache improves the average case (typically), but hurts the worst case (always) -- your worst case goes from "do expensive operation" to "do lookup in cache, fail, do expensive operation." Even if you parallelize misses ("do lookup in cache | do expensive operation, on cache success cancel expensive operation, on cache miss wait for expensive operation" you're generally adding at least a few operations in the critical path.
Maybe the takeaway is that this shit is hard and everyone is a little bit confused in their own way.
I'm not saying the only way to win is not to play, but I can neither confirm nor deny rumors that I may have at some point muttered this under my breath.