Comment by kev009 | Hacker News Reader

Comment by kev009 | Hacker News Reader

Parent Full thread

Yes as I said Varnish has great fit and finish. If you have a small pool of servers my comments will mean little to overarching business decisions.

To clarify my own part, this was in the back of my mind http://varnish-cache.org/docs/trunk/phk/notes.html. It's written sanctimoniously and is wrong. I care a lot about the profession of systems programming and hate to see people misled. If calling bullshit is trolling I guess everyone is either a bullshitter or a troll.

I contest most the points in that page. Varnish I/O model is ignorant of TLB flushing, instruction cache, VM hints like madvise(2), and the overhead (and downright difficult job) of an OS scheduler, as well as high performance networking KPIs like load balancing SO_REUSEPORT or RSS. In a production web cache there are very good reasons you might want to park some objects in memory outside the vfs (the vfs is demand paged and also carries heavy nontrivial locking). The stuff under the header "More caches" is correct, and trivially dealt with using per worker (cpu) APIs, see counter(9) in the FreeBSD kernel for a nice example.

By the text of the page, Varnish remains 2006 software. It's now 2018 and 48 hardware threads are common on servers, RAM is commonly measured in many hundred GB, and multicore programming is beginning to mature with high quality software, libraries, books, and tooling like Intel's VTune available.

If my points didn't have some validity, Fastly would not have done what they did. That's all I have to say. I don't know nor care about phk (and find the "smear me" comment hilariously paranoid), but a lot of the technical marketing is bullshit and these problems could be fixed if he were willing to back down and revisit the bold claims.

This is another misinformed rant.

> Varnish I/O model is ignorant of TLB flushing, instruction cache, VM hints like madvise(2), and the overhead (and downright difficult job) of an OS scheduler.

TLB flushing is a hardware problem, not a software problem. Instruction cache? Last I checked, the varnishd binary is about 2MB and uses very predictable jumping and straight function calling. Im pretty sure your going to be getting a 99% instruction cache hit rate. Madvise is used in Varnish's storage engine's, and there seems to be little interest or knowledge about how Varnish uses storage here.

> If my points didn't have some validity, Fastly would not have done what they did.

Can you expand on what Fastly did? They published a non locking memory allocator? So the claim is that memory allocation is the bottleneck and jemalloc is not suitable?

Sounds like someone might be stuck in performance hell and you are trying to draw parallels to Varnish which simply don't exist!

You don't understand TLB. It is distinctly not a hardware problem. A TLB is a content addressable memory and the mappings and flushing are done by.. software. You flush it every time you context switch, and pretty much fully all the time with KPTI now that PCID's 4096 ASIDs can't safely save anything other than the kernel and runnable process. My point about madvise protects against PHKs post claiming memory pool objects will always be paging in and out, which is false and again a memory pool is desirable for certain uses as it guarantees maximum latency and avoids very heavy locking in the VFS for high concurrency.

Well, if you want to "contest most the points in that page." I suggest you do so on substance, rather than by going after the person ?

There is a difference between "being ignorant" and "ignoring".

I don't think there is much in modern HiPerf computing I'm ignorant of (hint: Guess who's writing one of the most intense HiPerf scientific code right now), but there sure is a lot of it I'm deliberately ignoring with respect to Varnish.

TLB flushing, instruction caching ? When your CPU is 50+% idle most of the time, that's not really what (should) worry about.

And the "downright difficult job of an OS scheduler" isn't really that hard when all threads a waiting for I/O, wake up for a few microseconds, then go back to waiting for I/O again.

And so on, and so forth...

Again: Your total lack of actual experience running Varnish shows.

You should try it some day, you might actually find it useful :-)

You keep accusing me of logical fallacy but are using them yourself so I can't have a rational conversation with you. There is little CPU idle when you are doing TCP packet pacing, TLS in core, and edge compute. That's what I do on a 30Tbit/s CDN. I was just hoping you might reconsider the thread per connection but it appears not yet. Anyway congrats on the release and have a good day, sorry if the only takeaway you got from this is to be antagonized.

Thread-per-request seems even more correct to me, given the increased cost of system-calls, now that Spectre and Meltdown fixes are in.

I used to do thread-per-request, that has mediocre scaling at best. Even on the JVM this barely scales to a few thousand connections; native threads are heavier than that.

I've also done a lot of task-per-request (with each thread's affinity locked to a single hardware thread to avoid ripple effects), which does scale at least an order of magnitude more than threads.

I now use fiber-per-request for the best of both worlds: the easy sequential model of threads, yet the simple performance of tasks.

I can't understand why someone would defend the thread-per-request model at this point. It wasn't even a good model 10 year ago.

Correct

Isn't TCP packet shaping normally done on the network interface these days so you have CPU offloading?

s/shaping/pacing/, we're trying to smooth out the bursty nature of sending from high speed links and TSO so a flow is less likely to incur large contiguous buffer drops and tail drops along its path rather than limit the flow's throughput. I have a large fleet of intel NICs on the back half of their life cycle. NIC pacers sound good in theory and I'd like to eventually partial offload but they have limitations in terms of flows and number of pacing rates so will still require a software fallback. On a timerwheel, the system overhead for no offload and a partial offload is not nearly economical as theoretical full offload. And contrary to some misinformation from one of the varnish devs in this thread, taking an SWI/context switch/scheduler overhead (these basic concepts will probably get called "babble" by the dude) comprise much of the overhead.

Reply on news.ycombinator.com