Hacker News Reader

Comment by ngrilly | Hacker News Reader

Considering the latency between the user web browser and the server, does the “extremely low latency on account of caching all files in memory” really matter? The files should already be cached by the OS and using sendfile lets the OS copy the file directly to the network.

View on HN

I just figured any savings where savings can be made is a good thing, especially if there are zero downsides, plus it means very low CPU usage as it's literally just pulling from RAM without any CPU usage.

It also precaches gzipped and brotli'd versions of sensible files so that it can accept-encoding for a variety of browsers.

Couldn't the extra memory copies end up being an actual downside? I.e. the sort of thing that inspired sendfile(2) usage in some servers?

I’m not so sure, if you’re serving massive files this is not perhaps the option you want. If your SPA is so big you can’t handle a container that has its app code and assets in RAM perhaps it’s also too large.

I see it as a pragmatic trade off between speed and RAM. Even free tier EC2 instance should have enough RAM to handle pretty much any SPA even if the text content is duplicated (compressed) especially seeing as with a unikernel you have no RAM used up by Linux processes.

I think the argument is the same for files that fit in memory. This sort of thing is fairly well-trodden (if unsettled, to mix up the metaphors) ground, for yet another take there's varnish and Poul-Henning Kamp's strident writings about it.

For anyone else wanting to read about it: https://varnish-cache.org/docs/trunk/phk/notes.html

Linux already caches files in ram, you're probably just doing something that your OS is already doing, but worse.

If you're really concerned you can MMAP the file instead, and do a read every once in a while to keep it cached.

There are a lot of cases where this kind of functionality could cause big problems, and very few where it has more than a marginal improvement over what your OS is already doing. Just read the file every once in a while to keep it in memory cache.

This doesn’t necessarily have to run on Linux.

> serving things like Astro from S3

Given that point, I got the impression that the comparison was against loading resources from S3. The difference between that and serving from memory (even between that and serving from uncached traditional hard-drive) is going to be significant.

EDIT: cursory research suggests S3 latency (https://stackoverflow.com/a/861539/114292, being the most recent reference that turned up) is smaller than it was last time I remember needing to look into it, but still of the order of 10+ms.

Yeah I see this as a viable alternative to serving from S3 (I mean, which is still a great option I recommend provided you aren’t exposing S3 directly and set up S3 properly locked down) but the original niche was having an SPA web interface that runs on a k8s cluster as part of wider service where you want a tiny minimal and fast image that can be configurable when the pod is spun up. I just then expanded it to be a bit not of a general solution to have fun with it.

We deploy little apps internally. Users have low latency on the network - even home users we steer to use ISPs we know have low latency. I've found that speed can make a difference in user like for an app. Even going out to google etc I'm currently at 5-6 ms.

Some things we might have 1000+ checkboxs to work through on a checklist (cut down by some initial tailoring steps). Even things with a small wait on a click etc are frustrating. If you can drive using a keyboard shortcut and get very immediate response you can really move through things.

The difference of latency between serving a file cached in user-space RAM (what nano-web is doing) and serving a file from the kernel buffer cache (also in RAM) using sendfile should be totally negligible compared to the latency of your network, even if that network is "low latency".

Latency Numbers Everyone Should Know: https://static.googleusercontent.com/media/sre.google/en//st...

sendfile(2): https://man7.org/linux/man-pages/man2/sendfile.2.html

Latency between me and my cloud provider is about 10x lower than the latency of Python's http.server on localhost.

(It's negated by the SSL handshake, but that's another discussion...)

that sounds broken

Is that wall-clock measurement or are you using a tool like Apache Benchmark to see the latency?

I ask because those numbers seem really off.

The load time in Chrome Dev Tools

Once you add a CDN into the picture, it almost doesn't matter what you use to serve static assets for your web app.

Plus, like you said, 10-20ms extra on top of the actual latency from the server is not going to make any difference to the end user.

There are use cases where you probably need something like Nginx or whatever to handle lots of requests without a CDN but these are rare.

I agree on the CDN.

Reading assets from files instead of a RAM cache won’t add “10-20ms”. This is measured in microseconds, not milliseconds. Most servers nowadays are using SSDs or eNVM. And as I wrote earlier, the files are likely already cached in RAM by the OS anyway :)

> Reading assets from files instead of a RAM cache won’t add “10-20ms”.

I know. I was just using a hypothetical worst case scenario to exemplify that it doesn't matter.

https basically makes sendfile unusable for that use case.

Reply on news.ycombinator.com