Comment by lee_ars | Hacker News Reader

Comment by lee_ars | Hacker News Reader

Parent Full thread

Author here. I settled on the haproxy/varnish/nginx sandwich through an accretive process that started back in 2012 with wanting to speed up nginx by taking advantage of scads of extra RAM on the box I was hosting with at the time (and also being entranced by all the cool header games one can play with Varnish). Fast forward to 2015 as LetsEncrypt started to be a thing, and HTTPS started the mainstream push from a nice-to-have to an ought-to-have feature. I was comfy with Varnish + Nginx and I wanted to keep my cache layer, but Varnish has no SSL termination capabilities. I looked at a few options, including Hitch, but decided on HAProxy because it was more widely used and therefore easier to google answers to questions. Everything worked, so I never re-evaluated until 2022 when I got curious about OLS.

As to why Varnish + Nginx versus just Nginx + FastCGI or proxy cache, I was chasing the performance dragon. AIUI, Nginx's fastcgi and proxy caches just store the static-ified versions of dynamic content in flat files, and those files are accessed via the file system, which means they get cached into RAM whenever the file system decides they get cached into RAM. I had the free RAM, and Varnish's RAM-first approach felt like it would be faster. Plus, this was all for personal hosting, and playing with Varnish was fun. (Or at least "fun.")

On the WordPress side, consider using Batcache to generate appropriate Cache-Control headers. While it’ll also cache into your object cache (memcache/Redis), you can disable or ignore that functionality, and then you get a quite good integrated system. We use it with CloudFront’s page caching.

Oh man, Batcache. Now there's a name I haven't heard in a long time. I'll take a look!

Feel free to send me an email if you run into any issues! :)

Thanks for the walkthrough.

One particular comment:

> "Nginx's fastcgi and proxy caches just store the static-ified versions of dynamic content in flat files, and those files are accessed via the file system, which means they get cached into RAM whenever the file system decides they get cached into RAM. I had the free RAM, and Varnish's RAM-first approach felt like it would be faster."

The pagecache should do a pretty good job holding file contents in memory for you! https://tldp.org/LDP/sag/html/buffer-cache.html

I haven't had to tune for this scenario, but I think this might be the one place where you want a high swappiness (and some swap spaces). This will let Linux move unused bits of memory into swap & let even more ram be used for pagecache.

Have you considered caddy?

No, I'm happy to retreat to the familiar, and nginx is pretty familiar. After having my tail feathers burned with OLS, I doubt I'll switch to anything else for the next several years without an extremely compelling reason.

Sure I'm not suggesting it now as a follow up, I just wondered if it was something you were aware of and dismissed for some reason over the last several years before opting to try OLS (which I hadn't heard of) instead; especially since you mentioned Let's Encrypt.

I'm curious, how many requests-per-second have you managed to get with Varnish and how many with your NGINX FastCGI / Proxy caching settings?

Makes sense, particularly with the “fun” bit.

In the future, a ramdisk is a great way to make files from disk fast in a manner where the caller is naive to this. For instance you could create a ramdisk and then give that path to nginx for the caching config.

If I were going to do that, I'd just use tmpfs — i'm lazy. But I'm under the impression that given how aggressive and intelligent file system caching is these days, ramdisks are almost never the correct solution because the kernel is waaaaaaaaaay better at figuring out whether or not to cache files than I am.

Same same. s/ramdisk/tmpfs

Reply on news.ycombinator.com