Varnish and Cache Poisoning | Hacker News Reader

Varnish and Cache Poisoning

info.varnish-software.com

26 pointsr4um10 comments

I always thought 404 responses would occupy trivial amount of memory on cache since they don't have a body. Ok, there is some overhead which might cause issues if there is too much 404. However, I think some guy who is trying to crawl all of your web site is a bigger problem if your web site is big. You can throttle him based on his IP and user agent but how do you differiante people who has same IP because they are behing a cooparete proxy?

I also thought cache poisoning means putting a bad response into a cache by sending a special request so that innocent people will be served malicious response because they hit the cache. I think article uses it in a different context.

When was the last time you saw a 404 without a body?

Well, you are right, for most sites 404 returns a friendly error page. However for our case, it is a lot smaller page compared to 200 responses.

There's also the small matter that the pages tend to be "oddly" consistent.

I also thought cache poisoning means putting a bad response into a cache...

I agree. The cached responses mentioned in this article are perfectly valid; they're simply not very useful.

Rather than cache poisoning, I'd call this cache constipation.

I can't imagine PHK making such a basic mistake, who wrote the post?

The HTML header says

who according to crunchbase is a "Technical Presales Engineer" at Varnish.

Sorry for not misunderstanding. I'm not saying that PHK made a mistake, he picked the simple solution (which is very often the case). But LRU being too simple, you need to do more to fight again cache pollution.

Segmenting your storage is the solution I suggest in the blog post.

Let's call that cache pollution or flooding attacks. In the end, I consider it poisonous because I don't want to evict relevant content for junk (especially from the long tail).

I do mention that the RFC says that 404 responses can be cached, and that according to my own cache policy, I disagree.

It depends on your goals.

tl;dr: it turns out that OS-native caching policies aren't really such a great fit for edge caches, varnish should probably have a specific VCL module for intelligently manage cache poising attacks (even something simple for compactly managing 4XX errors would go a long way).

Reply on news.ycombinator.com