Mastering HTTP Caching – from request to response and everything | Hacker News Reader

Mastering HTTP Caching – from request to response and everything

blog.fortrabbit.com

134 pointsSujan23 comments

Article doesn't mention there are strong and weak ETags, only presenting the strong variant (those hashes in double-quoted strings).

Weak ETags are rare(r) beasts that look like `ETag: W/"foobar"` and mean that while response may be not byte-to-byte identical (e.g. it's a dynamically generated JSON document), it's still semantically the same - which can be useful for APIs (if caches implement all the logic correctly, of course).

This is important, especially when compression is part of the picture. For example, NGINX automatically replaces strong ETags with weak ones if gzip compression is enabled. Something to be aware of, if you're using them.

This is really good. Having worked at news sites and for a CDN its kindof fascinating how most web developers understanding of http basically ends at "GET or POST".

IMHO this stuff is increasingly becoming table-stakes for a successful online business. If you're not caching and/or serving your app in some kind of geographically distributed way, you're handing hundreds of milliseconds of performance delta to your competition. If you're nj.com then sure you can get away with serving everything from secaucus, but if you're any random sass app that even just wants to target the greater english speaking anglosphere of both brits and aussies you need to solve for this. Hell even the performance delta between the east and west coasts of the US can have a meaningful impact on ecommerce shopping carts.

I agree with everything you're saying, apart from this:

>performance delta between the east and west coasts of the US can have a meaningful impact on ecommerce shopping carts

Assuming a mid-sized, US sales only, ecom site, I would think serving it up somewhere in the geographic center (Dallas?) would suffice fine. Things like lazy loading images on category pages tend to make a bigger performance impact than a CDN in this restricted use case.

Chart that illustrates latency with US city pairs: https://i.stack.imgur.com/0pP3S.png

You are correct that zoomed all the way into single packet RTT you could split the difference with servers in texas. And if your shopping cart experience was somehow single http request-response steps with everything else async/lazy-loaded in a way that had no impact on the experience.

But in real world scenarios, open up the the chrome devtools network tab and click your way through a checkout process on any given ecom site. You'll probably make well north of 1000 requests overall, and even if a ton of it is fluff there's a solid 5 - 10 requests that are the blocking core of any page.

And thats just a single-user best case scenario example. When you zoom out a bit to the aggregate user base, a 5-hop/<100km-away network connection is going to have much less packetloss, jitter and latency than a 15-hop/>2000km one, such that the 95th percentile experience will be much better for the former.

All you can edge cache, though, is static content. Once the end user makes a product comparison, puts something in the cart, etc, it's no longer cacheable. That page-blocking stuff in the checkout path isn't usually able to be served by the CDN.

And, for content like javascript and css, that will be cached in the browser after the first request. Shopping cart software tends to use the same js/css on every page, with a few exceptions. The CDN, then, tends to serve up mostly product images. Lazy loading on category pages fixes the most pathologic problem there, if you don't have a CDN.

Just my real world experience that solely for this restricted use case, a CDN doesn't add much. Other optimizations matter more.

Edit: Not saying a CDN wouldn't be useful at all. But, if you're starting with typical cart software, like Magento, there are probably 100 other things that you should optimize before you worry about a CDN. We're able to get our ecom pages down to an average of 14 requests/page, many of those 14 being cachable at the browser and shared across pages. If you have customers more broadly spread, a CDN would move up the list quickly.

All you can edge cache, though, is static content.

That's generally true.

This was one of the problems I worked on at NuevoCloud.. for dynamic pages, it would look at the page, and determine which sections of the page are static or dynamic/unique. A PoP near the server would do the analysis, and then send the unique sections to the edge PoP (near the visitor) for it to be reassembled.

So the static portions of a page would be cached, and you would only be sending the unique parts of a page over long distances... and that would also be compressed, encrypted, and sent over an optimized connection (ie: no handshake delay, etc).

Of course, the difference is more significant the further away you are from the origin.

I don't know of any other CDNs doing that.. but I imagine they'll eventually get to it. So I would look for that in the future, even if you don't use our CDN.

That sounds like Raygun from Cloudflare and ESI (potentially) from Fastly.

https://www.cloudflare.com/docs/railgun/intro.html

https://www.fastly.com/blog/using-esi-part-1-simple-edge-sid...

ESI isn't from fastly.. it's an old W3C standard from 15 years ago. You have to markup all of your HTML to use it though.

So Cloudflare made a nice improvement with Railgun, so that you don't need to markup your HTML.. but you have to setup and run railgun on your server, so it can compare the pages. (I don't think it does anything else other than basically diving up the page content so individual parts can be cached.. but someone from cloudflare is welcome to tell me I'm wrong.)

To compare our tech to railgun:

- there's nothing to setup or run (you just enable it for the zone), and you don't need to modify your pages (like you do for esi)

- it works on all content, including binaries/images/etc

- each page/file only needs to be analyzed once. So if a page is first seen in a request from Japan, a 2nd request from Germany would be accelerated, even though that PoP has never seen the page before.

- Because of the way it works, it's paired with adaptive routing, which routes the data over the fastest active link between the pops.. so there's no handshake setup time, ssl negotiation time, etc over the long haul.

When its enabled, we actually use it to transmit most content between the pops.. so it also accelerates revalidation of static content, etc.

Edit: also it's not $200/mo.. it's just included with your account.

This article left me wanting, in terms of understanding why (or when) you'd want to have the server or proxy cache return a 304 (Not Modified) - I've been operating under the understanding that "the best request is one that is never made".

Given a complex web page with a large number of HTTP requests (especially now that bundling of assets is considered an anti-pattern with HTTP/2), that the latency introduced by all those repeated requests that end up returning a 304 are redundant if we can serve them directly from the client. Most caching tutorials seem to skip this area.

It doesn't make as much sense for a webpage, but a 304 can be effective for an API.

Say you have a resource that needs to be checked periodically, returning a 304 still has the overhead of a tcp connection but not the weight of the payload.

Thankfully, this treats the subject of revalidation, as opposed to simple caching setups/write-ups which merely set expiry periods for cached resources. Varnish (considered otherwise a competent cache) didn't handle it at all before V4.

[1]: http://stackoverflow.com/questions/34938706/varnish-4-does-n...

The bigger problem with varnish, to me, is that it doesn't support TLS. So you end up fronting it, with say nginx...which also has a reasonable cache, so why bother?

Edit: Should mention that varnish is a nice piece of software...not bashing. It's just that most people would prefer less pieces in their overall solution.

The varnish docs have a couple of philosophical essays on why they don't support SSL:

https://www.varnish-cache.org/docs/trunk/phk/ssl.html

https://www.varnish-cache.org/docs/trunk/phk/ssl_again.html

I think they are well justified in enforcing the separation of SSL code from caching memory management.

And no support for HTTP/2 until recently. They're suggesting Hitch for SSL termination now.

[1]: https://info.varnish-software.com/blog/varnish-cache-5-http2...

Edit: Apache also has caching/does it all in one package; they're even contemplating a module for certbot (in addition to or as replacement for the official certbot/letsencrypt Python package for Apache)

Is nginx as a reverse proxy as efficient at caching as varnish? Asking as I haven't found any benchmarks on the subject.

I'm using varnish via Fastly, and have come to greatly appreciate the fast cache-clear, so we can push all content out into the cache, not just static assets.

I believe NGINX's caching is as efficient, if not moreso, but it is less flexible. The tools available to you in Varnish's VCL are substantial, plus you get ESI support - something NGINX lacks.

Also selective cache purging is a paid feature with nginx.

If you're using the filesystem backed cache with nginx, selective cache purge, stats, etc, is a fairly trivial scripting task.

Agreed, There is also an OS alternative which essentially mirrors the functionality of propriety nginx cache purge: https://github.com/FRiCKLE/ngx_cache_purge

Hence you can just build it with your standing nginx binary reducing the need for miscellaneous scripts.

thanks for the info/link!

I doubt there's a notable performance difference, and it's a bit of apples and oranges unless you front varnish with something that does SSL for your benchmark. Not many use cases for plain http caching anymore.

Varnish does likely have an edge with flexibility and expressiveness around configuration, since it's a single purpose tool.

also note that by default most browsers will cache requests for days and will not hit the server. so remember to add the expire header or meta. and set it low for dynamic content.

Reply on news.ycombinator.com